Skip to content

Extending Webperf Core yourself

The sixteen tests cover what most websites have in common. But some requirements are your own: an internal coding standard, a rule that only applies to your CMS, a policy that should hold across every client site. You can add those yourself.

This is the most advanced page in the documentation. Most people never need it. But those who do tend to need it properly.

Do you actually need your own test?

Ask first whether one of these is enough:

  • Settings. Several tests can be steered without code. See configuration.
  • A subset of the existing tests. Run only the ones measuring what you care about: -t 22,9,18.
  • The review rather than the score. With -r you get a list of concrete problems, which you can process with your own tooling.
  • The raw data. --setting general.review.data=true gives you the assessment as JSON, and -o result.json saves the whole result. It's often simpler to analyse the output than to build a new check into the tool.

Is there something left that no existing check will spot for you? Then a test of your own is the right move.

Two ways to extend

A standalone test

Makes its own requests. Suits auditing DNS, email, response headers or anything else that doesn't need the page rendered in a browser. Written in Python, in tests/.

A Sitespeed.io plugin

Runs as part of the shared Sitespeed run and gets access to the rendered page. Suits cases where you need the finished content, and means your check costs no extra page load. Written in JavaScript and published as an npm package. See plugin-standard-files and plugin-css for working examples.

The rest of this page is about standalone tests, which is the more common route.

Keep the change, or contribute it?

Two quite different levels of ambition, and it pays to choose early.

Requirement Your own use Contributing upstream
Test number Any free one Coordinated with the project
Translations Just your own language Every language the project supports
Help text Nice to have Required
Regression test Optional Required
Maintenance Yours, at every update The project's

Your own fork costs maintenance

If you add a test to your own copy, you get to merge your changes into every new version of Webperf Core. If the check is useful to more people than you, it's almost always cheaper to contribute it and be rid of the maintenance.

Step by step

1. Claim a test number

Add a constant at the end of the tuple in helpers/test_helper.py and increment range():

TEST_ALL = (...,
            TEST_PRIVACY,
            TEST_DNS,
            TEST_MY_TEST
            ) = range(34)

Never insert the number in the middle of the list. The position is the test number. Everything after it would shift and change meaning in historical results. See architecture.

2. Register the function

Your test goes into TEST_ALL_FUNCS and, if it runs standalone, into TEST_FUNCS:

from tests.my_test import run_test as run_test_my_test

TEST_ALL_FUNCS = {
    ...
    TEST_MY_TEST: run_test_my_test
}

3. Write the test

Create tests/my_test.py with a run_test:

def run_test(global_translation, url):
    """
    Returns a tuple of Rating and raw data.
    """
    local_translation = get_translation(
        'my_test',
        get_config('general.language'))

    rating = Rating(
        global_translation,
        get_config('general.review.improve-only'))

    # ... do your check ...

    return (rating, result_dict)

If the website can't be reached, return a Rating with overall_review set to TEXT_SITE_UNAVAILABLE rather than letting an exception bubble up. A test that crashes stops the whole run for that website, including the tests that would have worked.

4. Set the score

Use Rating and fill in only the sub-scores your test actually measures: overall, integrity and security, performance, accessibility or standards compliance. A test that says nothing about performance shouldn't set a performance score. Otherwise it drags down a sub-score it never measured.

5. Add translations

Create locales/en/LC_MESSAGES/my_test.po and the equivalent for whichever languages you need. Every string shown to the user must go through the translation. See translations.

If you're building for your own use, your own language is enough. To contribute upstream, every language needs filling in.

6. Add settings

Defaults belong in defaults/settings.json under tests.my-test., and are read with get_config('tests.my-test.key'). That way your test can be steered with --setting and settings.json like every other.

7. Add help text

The test should appear when someone runs python default.py -t ?. Add a TEXT_TEST_VALID_ARGUMENTS_MY_TEST string to the language files and print it in show_test_help in default.py.

The locale files are the authority for test names

That string isn't just help text. It's the test's canonical name, and it's what other systems read to display it. Among them /v1/tests on api.webperf.se, and this documentation. Word it the way you want the test to be called.

8. Write a regression test

Add a workflow under .github/workflows/ following the pattern of the existing ones. It should run the test against a known website and compare the result. Required for contributions upstream. It's a good idea for your own use too. That's how you find out an update to Webperf Core changed your test's outcome.

See also