Theme: Testing

[gabestein]

Summary

We're making incredible strides shipping features quickly, but this is resulting in deploying too-frequent regressions and bugs into production. To reduce the impact of these regressions while maintaining our velocity, and to help users and developers be more confident in the app, we'll implement testing standards (both automated and QA) and feature flags.

Core Goals

• Reduce impact of regressions and bugs introduced to production
• Understand code coverage as a signal of confidence for users and other developers
• Agree on and implement testing (automated and QA) standards and processes

High-Level Projects

Note: I'm fairly out of my depth here, so this is my best attempt based on our prior discussions.

• Test suite that runs on deployment
• Automated code coverage?
• Feature flag system for launching experiments in production
• Deployment process that includes automated and manual testing standards

Technical implementation notes

[gabestein]

Need to brainstorm critical test paths to prioritize. Initial thoughts:

• Creating a pub
• Editing a pub
• Forms in general
• Permissions

[gabestein]

Measurement for regressions and bugs using existing tickets.

[gabestein]

PR deployment environment for QA

[allisonking]

These were the tickets we made from last time we discussed this—a good chunk of them are done!

Set up infrastructure to allow vitest tests to use a database
Add db fixtures to help seed data for tests
Allow seeding db from playwright
Provision a db for CI
Testing docker container with browser(s) to run playwright tests