Skip to content

Architecture

Folders

Folder Contents
tests/ One module per test, plus shared helpers in utils.py
helpers/ The test registry, settings handling and data models
engines/ Output formats: JSON, CSV, SQL, SQLite and Markdown
locales/ Translations, one directory per language
defaults/ The project's default settings
data/ Reference data, including IP2Location
docker/ Scripts for building and running the image

The entry point is default.py in the root.

A run, step by step

  1. default.py parses the command line in CommandLineOptions and loads the settings.
  2. helpers/test_helper.py translates the test numbers into functions. The registry is three lookup tables: TEST_ALL_FUNCS (what exists), TEST_FUNCS (what runs standalone) and TEST_USE_SITESPEED (what runs as a plugin inside Sitespeed.io).
  3. The test runs and returns a Rating object along with raw data.
  4. engines/ writes the result in the chosen format if -o was given.

The test numbers

The numbers are defined in helpers/test_helper.py as a tuple unpacked against range(33):

TEST_ALL = (TEST_UNKNOWN_00,
            TEST_DEPRECATED, TEST_PAGE_NOT_FOUND,
            ...
            TEST_PRIVACY,
            TEST_DNS
            ) = range(33)

The position in the tuple is the test number. That's why a retired test must never be removed from the list. Everything after it would shift, and historical results on Webperf.se would change meaning. Retired positions are filled with TEST_DEPRECATED instead.

Two kinds of test

Standalone tests make their own requests. HTTP, email, DNS and Webbkoll belong here.

Sitespeed-based tests run as plugins inside a shared run of Sitespeed.io. The 404 page, standard files, the accessibility statement, the three lint tests and Lighthouse belong here. Each has its own npm module, such as plugin-standard-files and plugin-css.

That's why a combined run is faster than the sum of the individual ones: the website only has to be loaded once.

Scoring

helpers/models.py contains Rating, which holds an overall score plus sub-scores for integrity and security, performance, accessibility and standards compliance. Each test fills in the parts it has something to say about, and calculate_rating in tests/utils.py weighs them together.

The general.review.improve-only setting controls whether the review includes what already works or only what could be better.

Reading further in the code

To see what a test looks like from the inside, start with tests/energy_efficiency.py. It's short, has no external dependencies, and shows the whole chain from measurement to score.

See also