Developer Guide

This section contains everything that a StrictDoc developer/contributor should know to get the job done.

StrictDoc is based on Python and maintained as any other Python package on GitHub: with linting, tests, and hopefully enough best practice put into the codebase.

The instructions and conventions described below are a summary of what is considered to be the currently preferred development style for StrictDoc.

Any feedback on this development guide is appreciated.

StrictDoc itself mostly depends on other Python Pip packages, so there is nothing special to be installed.

You may need to install libtidy if you want to run the integration tests. The HTML markup validation tests depend on libtidy.

On Linux Ubuntu:

sudo apt install tidy

From the core Python packages, StrictDoc needs Invoke, Tox and TOML:

pip install invoke tox toml

As reported by a user, Windows Long Path support has to be enabled on a Windows system.

You can find information on how to enable the long paths support at https://pip.pypa.io/warnings/enable-long-paths.

git clone https://github.com/strictdoc-project/strictdoc.git && cd strictdoc
pip install toml
python3 developer/pip_install_strictdoc_deps.py
python3 strictdoc/cli/main.py

The pip_install_strictdoc_deps.py installs all dependencies of StrictDoc, but not StrictDoc itself.

An alternative approach for those familiar with development in a virtual environment:

git clone https://github.com/strictdoc-project/strictdoc.git && cd strictdoc
python3 -m venv .venv
. ./.venv/bin/activate
pip install -e .
python3 strictdoc/cli/main.py

After this, running developer/pip_install_strictdoc_deps.py should report that all packages are already installed.

This installs all the default dependencies of StrictDoc. To also install the dev dependencies specified in pyproject.toml:

pip install ".[development]"

All development tasks are managed using Invoke in the tasks.py file. On macOS and Linux, all tasks run in dedicated virtual environments. On Windows, invoke uses the parent pip environment, which can be a system environment or a user's virtual environment.

Make sure to familiarize yourself with the available developer tasks by running:

invoke --list

Before doing anything else, run the main check command to make sure that StrictDoc passes all tests on your system:

invoke check

The check command runs all StrictDoc lint and test tasks with the only exception of end-to-end Web tests that are run with a separate task (see below).

• The version of Python is set to be as low as possible given some constraints of StrictDoc's dependencies. Ideally, the lowest Python version should only be raised when it is consistently deprecated by major software platforms like Ubuntu or GitHub Actions.
• All developer tasks are collected in the tasks.py which is run by Invoke tool. Run the invoke --list command to see the list of available commands.
• Formatting is governed by black which reformats the code automatically when the invoke check command is run.
  • If a string literal gets too long, it should be split into a multiline literal with each line being a meaningful word or a subsentence.
• Avoid shortening variable names unnecessarily. For example, use 'buffer' instead of 'buf', 'document' instead of 'doc', 'function' instead of 'func', 'length' instead of 'len', etc. Note: While some older parts of StrictDoc may not adhere to this guideline, they are planned to be refactored in the future.
• For "element is non-None" checks, a full form shall be used, for example: if foo is not None instead of if foo. This helps to avoid any confusion with all sorts of strings (empty or non-empty str, Optional[str]) that are used extensively in StrictDoc's codebase. The non-None and non-empty string check shall therefore be as follows: if foo is not None and len(foo) > 0. The explicit check also applies to any other kinds of objects besides strings: if foo is not None instead of if foo. Rationale: if foo makes it unclear whether the intention is to check is not None or also len(foo) > 0.
• For lambdas and short for loops, the recent convention is to add _ to the variables of a for loop or a lambda to visually highlight their temporary use within the current scope which is done to counter the fact that these variables can leak and be used outside of the scope of the loop. Example:

for foo_, bar_ in baz:
    # use foo_, bar_ within the loop.

• The function arguments with the default values shall be avoided. This convention improves the visibility of the function interfaces at the coast of increased verbosity which is the price that StrictDoc development is willing to pay, maintaining the software long-term. The all-explicit function parameters indication is especially useful when the large code refactorings are made.
• StrictDoc has been making a gradual shift towards a stronger type system. Although type annotations haven't been added everywhere in the codebase, it is preferred to include them for all new code that is written.
• For opening files, use the helpers file_open_read_utf8 and file_open_read_bytes. These helpers perform normal file opening but also strip the UTF-8 BOM character, which is added by some Windows tools.
• If a contribution includes changes in StrictDoc's code, at least the integration-level tests should be added to the tests/integration. If the contributed code needs a fine-grained control over the added behavior, adding both unit and integration tests is preferred. The only exception where a contribution can contain no tests is "code climate" which is work which introduces changes in code but no change to the functionality.

The preferred Git workflow is "1 commit per 1 PR". If the work truly deserves a sequence of commits, each commit shall be self-contained and pass all checks from the invoke check command.

For larger pull requests involving too many changes, the preferred approach: split the work into several independent PRs to simplify the work of the reviewer.

The branch should be always rebased against the main branch. The git fetch && git rebase origin/main is preferred over git fetch && git merge main.

The Git commit message should follow the format of Conventional Commits:

<type>(<optional scope>): <description>

where the scope can be a major feature being added or a folder.

A form of <type>(scope): <subscope>: <description> is also an option.

Examples:

feat(html2pdf): add a new option to force page breaks
fix(backend/sdoc_source_code): add Rust support
refactor(cli): migrate "import excel" to command pattern
chore(cli): rename shared.py -> _shared.py
chore(.github): switch the macOS tests to macos-latest
docs: update release notes

If the committed work is dedicated to more than one topic, use comma-separated scopes. Example:

refactor(server, UI): update to new requirement styles

The shortest path to run the server when the StrictDoc's source code is cloned:

invoke server

invoke test-end2end

The integration tests are run using Invoke:

invoke test-integration

The --focus parameter can be used to run only selected tests that match a given substring. This helps to avoid running all tests all the time.

invoke test-integration --focus <keyword>

See How to test command-line programs with Python tools: LIT and FileCheck to learn more about LIT and FileCheck, which enable the StrictDoc integration tests.

• Every change in the functionality or the infrastructure should be documented.
• Every line of documentation shall be no longer than 80 characters. StrictDoc's own documentation has a few exceptions, however, the latest preference is given to 80 characters per line. Unfortunately, until there is automatic support for mixed SDoc/RST content, all long lines shall be edited and split by a contributor manually.
• The invoke docs task should be used for re-generating documentation on a developer machine.

• snake_case everywhere, no kebab-case.
  • This rule applies everywhere where applicable: file and folder names, HTML attributes.
  • Exception: HTML data-attributes and testid identifiers.

• Add to the release notes (docs/strictdoc_04_release_notes.sdoc) a high-level, human-readable summary of all work since the latest Git tag.
• Summarize all changes under the top section called "Unreleased". If that section does not exist yet, create it. It is this section that will later be turned into the actual release notes.
• The section shall start with This release contains the following enhancements:<full empty line break>.
• Follow the example of the existing Release Notes.
• Do not summarize each and every commit. Only include the most important user-facing changes that are implemented with feat: and fix: commits. Don't consider commits with any other prefix.
• Ignore the changelog file CHANGELOG.md completely. It is an outdated auto-generated file that is going to be removed in the future.
• Do not try to create a new version yourself. This is handled with another playbook (TBD).
• Do not modify any existing content from the previous releases.