Skip to content

Instantly share code, notes, and snippets.

@dominikkaegi

dominikkaegi/e2e-testing-documentation.md

Last active February 22, 2021 15:52
Show Gist options
  • Save dominikkaegi/9448fb752fa916f117c5fcfe39d4a2e9 to your computer and use it in GitHub Desktop.
Save dominikkaegi/9448fb752fa916f117c5fcfe39d4a2e9 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
e2e-testing-documentation.md

E2E Testing With Cypress

Goal of E2E testing

  1. In a first step we aim to automate the smoke tests we run before each release.
  2. In a second step the goal is to increase the scope of the tests to also run with mock data so that we can simulate all kind of states in the FE application and test if the FE behaves correctly. And example for mocked state could be a request failing and making sure the correct error state is displayed on the FE.
  3. In a last step we want to add visual regression tests.

Test Setup

Folder Structure

e2e
	- cypress
		- config			// configuration for different environments
			- demo.json
			- localhost.json
		- fixtures			// mock data
			- example.json
		- integration 		// the tests
			- flows
				- dashboard-widgets.e2e.ts
				- login.e2e-spec.ts
				- super-admin.e2e.ts
		- plugins
			- index.js
		- support			// e2e custom commands (e.g. testinglibrary, cy.login etc.)
			- commands.ts
			- index.d.ts
			- index.js
		- videos			// the recorded videos
			- flows
				- dashboard-widgets.e2e.ts.mp4
	- cypress.json
	- package.json
	- tsconfig.json

Configurations

The shared configuration are in the cypress.json file. If we need configurations for a certain environment we added it to the configuration sin the config folder.

Fixtures

Mock test data which we need for our tests. Checkout the Fixture Docs on how to use them.

Integration

Integration are the folder in which we add our integration tests

Plugins

A folder in which we can add plugins. For now we only load different configurations based on the environment we are running cypress in. There is a List of official and community plugins through which we can add additional functionality to cypress.

Support

We can define our own commands. In this section we are for example adding the @testing-library/cypress commands so that we can use them throughout our tests. If we are adding our own custom commands, for example our login command, we need to add our own type definition in support/index.d.ts.

Running the Tests

You can run the tests in the open mode or in a headless mode

  • yarn e2e:run run in headless mode on demo1
  • yarn e2e:dev run in open mode on localhost
  • yarn e2e:demo run in open mode on demo1

Test Users

Username Password Description
ND1043 nurfuerdentestroboter The default user for e2e tests when using the login command.
NN1008 nurfuerdentestroboter Used for the language change test
YW2729 nurfuerdentestroboter
HJ6711 nurfuerdentestroboter
ND1043 nurfuerdentestroboter

Cypress Dashboard

The cypress dashboard allows us to collect data over the different e2e test runs.

dashboard.cypress.io

You need to have an invite to access the dashboards.

Pipeline

The pipeline integration does not exists as of right now. How we will incoporate it into the pipeline is open for discussion. For a start the suggestion is to run the pipeline against demo1 once a day. In a next step we could think of running it after each merge request to master.

Finding a semantic Selector

Browser Extension

We are using when possible the selectors from @testing-library/cypress. To find a good selector there is a browser extension which will help you out called Testing Playground. The extension is written for finding elements for unit tests, so it won't give you the async findBy selector. For example it will give you the following selector

screen.getByRole('button', {
  name: /benutzer erstellen/i
})

Which we then change to the cy selector

cy.findByRole('button', {
	name: /benutzer erstellen/i
})

Using Regex to Find By Text

Whenever possible use a regex selector with the ignore case flag. That makes our selectors a little more resilient to changes of the text. However there are cases in which a regex targets multiple elements, in that case use a more specific text selector.

For example in the create user form in the super admin section we have a field for Vorname and a field for Zweiter Vorname for which the regex /Vorname/i will target both elements. Thus we will use the exact string to select the first element.

cy.findByLabelText('Vorname').type(mockUser.firstName);
cy.findByLabelText(/zweiter vorname/i).type(mockUser.middleName);

Selector Priority ⭐️

To follow the guideline of selecting elements like a user would we try to follow a certain priority when using the different selectors. Find the priority list in the testing-library docs: selector priority

FindByRole

Find an element by its role and text description.

cy.findByRole('button', {
	name: /benutzer erstellen/i
}).click()

FindByLabelText

Every element in a form should have a label text. Therefore you can select every element by its label text. if it does not have a label text it either is not an accessible form or the label and the input are not correctly linked. Fix the link or add an element. If a label should not be visible for the element you can add the label with the aria-label property on the element.

cy.findByLabelText(/Bearbeiten/).click();

Limiting the selector space

When try to select an element within the whole dome, there is a change that we might find two elements for the same selector. That can make selecting the correct element hard.

The solution is to limit the space within which we search the selector. We have following options:

Navigation

cy.get('nav').within(() => {
	// Select Element within the navigation
})

Main

cy.get('main').within(() => {
	// Select Element within the main content
    });

Modal

cy.findByRole('dialog').within(() => {
	// Find Element within a modal
});

Listbox (Multi Select)

cy.findByRole('listbox').within(() => {
    // The multi select adds HTML on the root level thus we can find the elements
    // with searchint for an element with the role listbox on the root level
    // The current listbox implementation currently requires the {force: true}
    // option on clicks.
});

Improve The Semantic Structure of the HTML

If you can not find a semantic selector for an element, can you improve the semantic of the html? Often you can improve the semantics of the html with a few small changes.

Adding a section (header, footer)

The elounge does not have the most semantic HTML structure. Often we can improve the semantics of the html and make the selection easier for us by using section. This often just means changing a div element to an section element. Within that section we can also have its own header and footer element. That can make the selection easier for sub elements within that secion. A section usually also have a title. By connecting the section with its title with the aria-labelledby element you can make the selection of the section a breeze.

Adding a label / labelled-by / aria-label

Does a label already exist but is not connected to your element? Look at the screen and the DOM to see if there is already a label for your element but it is just no connected correctly. Fix the connection and select the element by label.

For example, a section often has a title within the section, you can link it up by creating an id for the title and add the labelled-by on the section. That way you can easily select the whole section with a findByLabelText.

Example:

<div className="dashboard-add-widget-modal__widget">
    <div className="d-flex flex-column justify-content-center mr-3">
        <div className={classNames('dashboard-add-widget-modal__widget-image', `dashboard-add-widget-modal__widget-image--${widgetType}`)} />
    </div>
    <div className="dashboard-add-widget-modal__widget-description">
        <strong>{t(`cmb-web.dashboard-add-widget-modal.widget.${kebabCase(widgetType)}.title`)}</strong>
        <br />
        <small className="text-muted">{t(`cmb-web.dashboard-add-widget-modal.widget.${kebabCase(widgetType)}.description`)}</small>
    </div>
    <div className="dashboard-add-widget-modal__widget-action d-flex justify-content-center flex-column ml-3">
            <div className="text-muted small text-right">{t('cmb-web.dashboard-add-widget-modal.maximum-instances-per-view-reached')}</div>
        )}
    </div>
</div>

Increasing the semantic of the html by

  • replacing div with section
  • the strong with the h3
  • adding a labelled-by to the section and connect it with its title
<section className="dashboard-add-widget-modal__widget" aria-labelledby={titleId}>
    <div className="d-flex flex-column justify-content-center mr-3">
        <div className={classNames('dashboard-add-widget-modal__widget-image', `dashboard-add-widget-modal__widget-image--${widgetType}`)} />
    </div>
    <div className="dashboard-add-widget-modal__widget-description">
        <h3 id={titleId}>{t(`cmb-web.dashboard-add-widget-modal.widget.${kebabCase(widgetType)}.title`)}</h3>
        <small className="text-muted">{t(`cmb-web.dashboard-add-widget-modal.widget.${kebabCase(widgetType)}.description`)}</small>
    </div>
    <div className="dashboard-add-widget-modal__widget-action d-flex justify-content-center flex-column ml-3">
            <div className="text-muted small text-right">{t('cmb-web.dashboard-add-widget-modal.maximum-instances-per-view-reached')}</div>
        )}
    </div>
</section>

Non Semantic Selector Exceptions

You might run in situations where a larger code refactor might be needed to create semantic selectors for an element. In that case use a non semantic selector and add a comment on the reason why you chose a non semantic selector.

Interacting With the Dom

  • .click()
  • .dblclick()
  • .righclick()
  • .type()
  • .clear()
  • .check()
  • .uncheck()
  • .select()
  • .trigger(), trigger an specific event e.g. .trigger('mousedown')

Click is not working

Cypress by default only clicks and element if the DOM is "ready to receive" actions. Read more on that in the cypress docs.

This means even if the element seems to be on the page cypress might not click on because of that definition. It can also happen that during a test another action within the application triggers a toast notification which could place an element infront of your element you would like to click. For these cases we can tell cypress to ignore the definition of actionalibility by adding the parameter {force: true}.

  • Use cy.click({force: true})

Select is not working

Selecting an element only works if the element to select only exists once as an option. For example in the country list we have Switzerland twice, once in the first five quick options and a second time in the list. Cypress currently does not support this kind of scenario.

Test Approach

Clean Up

If possible create E2E test in a way which cleans the data up again before it finished.

Clean up state in beforeEach before each and not afterEach. Because there is no guarantee that the afterEach will run.

Conditional Testing

If you have a need for coniditional testing, do not depend on the DOM to verify if the condition is met. The reason is that the DOM is unstable and your value might appear within 100ms, but for a computer that 100ms does not matter. The test runner is so fast, that it will think the value does not exist and thus not run the test. If you run the test you need to have some kind of deterministic source, for example you can make a request and decide on the data you receive if a certain case should run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment