easy-server - Secure server access that is easy to use¶

The easy-server package is a Python library for securely defining sensitive information for accessing servers (or services), such as logon credentials or API keys.

The information for accessing the servers is divided into a general portion that is defined in an openly accessible server file, and a sensitive portion that is defined in an encrypted vault file.

The vault file defines the secrets needed to access the servers, such as logon credentials or API keys. The vault file must be an “easy-vault” file and thus can be encrypted and decrypted using the easy-vault command provided by the easy-vault package. The “easy-vault” files remain encrypted in the file system while their content is used to access the servers.

The server file defines general information about the servers, such as a short description, contact name, or a reminder which network to use for accessing them.

The link between the server file and the vault file are user-defined nicknames for the servers. These nicknames can also used by users as a convenient way to identify servers in commands.

The server files support the definition of server groups that also have a nickname.

Typical use cases for the easy-server package are test programs running end-to-end tests against real servers, or command line clients that access servers or services.

This provides a convenient, flexible and secure way how Python programs can retrieve the secrets needed for accessing servers or services, while protecting these secrets in a secure way.

Usage¶

Supported environments¶

The easy-server package is supported in these environments:

Operating Systems: Linux, macOS / OS-X, native Windows, Linux subsystem in Windows, UNIX-like environments in Windows.
Python: 2.7, 3.4, and higher

Installation¶

The following command installs the easy-server package and its prerequisite packages into the active Python environment:

$ pip install easy-server

Server files¶

A server file contains the openly accessible portion of the servers and optionally references a vault file that specifies the secret portion of the servers.

The server file must be in YAML syntax and must follow the rules described in this section.

The server file defines servers, server groups and a default server or group. The servers and server groups are identified using user-defined nicknames and the server file stores some basic information about them.

The vault file is optional and its path name is specified with a property in the server file. Corresponding server items in these two files have the same nicknames. See Vault files for details.

Here is an example server file that defines two servers and one server group, and additional user-defined properties named ‘stuff’:

vault_file: vault.yml                 # Relative to directory of this file

servers:                              # Fixed top-level key

  myserver1:                          # Nickname of the server
    description: "my dev system 1"    # Short description of the server
    contact_name: "John Doe"          # Optional: Contact for the server
    access_via: "VPN to dev network"  # Optional: Any special network access needed
    user_defined:                     # Optional: User-defined additional information
      # User-defined properties:
      stuff: morestuff

  myserver2:                          # Nickname of the server
    description: "my dev system 2"    # Short description of the server
    contact_name: "John Doe"          # Optional: Contact for the server
    access_via: "Intranet"            # Optional: Any special network access needed
    user_defined:                     # Optional: User-defined additional information
      # User-defined properties:
      stuff: morestuff

server_groups:                        # Fixed top-level key

  mygroup1:                           # Nickname of the server group
    description: "my dev systems"     # Short description of server group
    members:                          # Group members (servers or groups)
      - myserver1
      - myserver2
    user_defined:                     # Optional: User-defined additional information
      # User-defined properties:
      stuff: morestuff

default: mygroup1                     # Fixed top-level key: default server or group

In the example above, myserver1, myserver2, and mygroup1 are nicknames of the respective server or server groups. These nicknames are used when servers or groups are put into a server group in that file, or when they are specified as a default in that file, or when they are used in functions of the easy-server package. See easy_server.ServerFile for details.

These nicknames are case sensitive and their allowable character set are alphenumeric characters and the underscore character, i.e. A-Z, a-z, 0-9, and _.

The value of the optional vault_file top-level property is the path name of the vault file that belongs to this server file. Relative path names are relative to the directory of the server file.

The value of the servers top-level property is an object (=dictionary) that has one property for each server that is defined. The property name is the server nickname, and the property value is an object with the following properties:

description (string): Short description of the server (required).
contact_name (string): Name of technical contact for the server (optional, defaults to None).
access_via (string): Short reminder on the network/firewall/proxy/vpn used to access the server (optional, defaults to None).
user_defined (object): User-defined details of the server (optional). Can be schema-validated via the user_defined_schema init parameter of easy_server.ServerFile.

The value of the server_groups top-level property is an object that has one property for each server group that is defined. The property name is the group nickname, and the property value is an object with the following properties:

description (string): Short description of the server group (required).
members (list): List of server nicknames or other group nicknames that are the members of the group (required).
user_defined (object): User-defined details of the group (optional). Can be schema-validated via the group_user_defined_schema init parameter of easy_server.ServerFile.

The value of the default top-level property is a string that is the nickname of the default server or group.

Server groups may be nested. That is, server groups may be put into other server groups at arbitrary nesting depth. There must not be any cycle (i.e. the resulting graph of server groups must be a tree).

A particular server or server group may be put into more than one server group.

Vault files¶

A vault file contains the sensitive portion of the servers, such as passwords or API keys.

The vault file must be an “easy-vault” file and can be encrypted and decrypted using the easy-vault command provided by the easy-vault package.

The “easy-vault” files must be in YAML syntax and must follow the rules described in this section.

Here is a complete working example of a vault file that defines host, username and password for the servers from the example server file shown in the previous section:

secrets:                                # Fixed top-level key

  myserver1:                            # Nickname of the server
    # User-defined properties:
    host: "10.11.12.13"
    username: myuser1
    password: mypass1

  myserver2:                            # Nickname of the server
    # User-defined properties:
    host: "9.10.11.12"
    username: myuser2
    password: mypass2

The vault file must have one top-level property named secrets. Below that are properties that represent the servers (or services).

The server items are identified by nicknames (myserver1 and myserver2 in the example above) and can have an arbitrary user-defined set of properties (host, username and password in the example above). The properties may be of arbitrary types, i.e. you can build substructures as you see fit.

The values of the server items can be schema-validated via the vault_server_schema init parameter of easy_server.ServerFile, or the server_schema init parameter of easy_server.VaultFile.

Here is another example that defines URL and API key for the servers (or rather for the services, in this case):

secrets:                                # Fixed key

  myserver1:                            # Nickname of the server
    # User-defined properties:
    url: https://10.11.12.13/myservice
    api_key: mykey1

  myserver2:                            # Nickname of the server
    # User-defined properties:
    url: https://9.10.11.12/myservice
    api_key: mykey2

Because the server file has user-defined properties for each server entry, and the structure of the server entries in the vault file is user-defined, there is a choice of which information is put into which file. For example, the host property from the previous examples could have been moved into the server file as a user-defined property, since usually it is not really a secret.

The vault file can be encrypted or decrypted using the easy-vault command that is part of the easy-vault package

The vault file can be in the encrypted state or in clear text when the easy-server library functions are accessing it. It is recommended to always have it in the encrypted state and to decrypt it only for the period of time while it is edited.

Example usage¶

The following code snippet shows how a server file and a vault file is used to get to all the information that is needed to access a server or in this example, the servers in a server group:

import easy_server

# Some parameters that typically would be input to the program:
server_file = 'examples/server.yml'      # Path name of server file
nickname = 'mygroup1'                    # Nickname of server or group

try:
    esf = easy_server.ServerFile(server_file)
except (easy_server.ServerFileException, easy_server.VaultFileException) as exc:
    print("Error: {}".format(exc))
    return 1

es_list = esf.list_servers(nickname)  # Works for server and group nicknames

for es in es_list:
    nick = es.nickname

    # The structure of the secrets in the vault file is user-defined.
    # Here, we use the first example vault file.

    host = es.secrets['host'],
    username = es.secrets['username']
    password = es.secrets['password']

    print("Server {n}: host={h}, username={u}, password=********".
          format(n=nick, h=host, u=username))

    # A fictitious session class
    session = MySession(host, username, password)
    . . .

Example usage with schema validation¶

The following code snippet is an extension of the previous example that shows how the user-defined items in a server file and the server items in a vault file are validated with JSON schema.

Note that JSON schema validates the structure of complex objects and its use does not require that the input data that is being validated is in JSON syntax. In fact, in our case the input data is in YAML syntax, and what is validated is the complex object representing the YAML file after it has been parsed.

For details about how to define a JSON schema, see https://json-schema.org/understanding-json-schema/. Note that the JSON schemas used in this project are represented using a Python dict object that represents the JSON schema (i.e. as if it had been loaded from a JSON schema file using json.load()). Due to the syntax similarities between JSON and literal dict specifications in Python, the JSON schema examples in that document can be specified directly in Python.

The server file used in this example is again the one shown earlier. In the JSON schema, its user-defined property ‘stuff’ is defined as string-typed and optional, and no additional properties are allowed. In this example, the user-defined portions of the server items and group items are defined to be the same.

The vault file used in this example is the one with the ‘host’, ‘username’ and ‘password’ properties shown earlier. In the JSON schema, these properties are all defined as string-typed and required, and no additional properties are allowed.

import easy_server

# Some parameters that typically would be input to the program:
server_file = 'examples/server.yml'      # Path name of server file
nickname = 'mygroup1'                    # Nickname of server or group

# JSON schema applied to the value of the user_defined element of each
# server item in the server file:
user_defined_schema = {
    "$schema": "http://json-schema.org/draft-07/schema#",
    "type": "object",
    "properties": {
        "stuff": { "type": "string" },
    },
    "required": [
        # 'stuff' is optional
    ],
    "additionalProperties": False,
}

# JSON schema applied to the value of the user_defined element of each
# group item in the server file. In this example, the same schema as for
# the server items is used.
group_user_defined_schema = user_defined_schema

# JSON schema applied to the value of each server item in the vault file:
vault_server_schema = {
    "$schema": "http://json-schema.org/draft-07/schema#",
    "type": "object",
    "properties": {
        "host": { "type": "string" },
        "username": { "type": "string" },
        "password": { "type": "string" },
    },
    "required": [
        "host",
        "username",
        "password",
    ],
    "additionalProperties": False,
}

try:
    esf = easy_server.ServerFile(
        server_file,
        user_defined_schema=user_defined_schema,
        group_user_defined_schema=group_user_defined_schema,
        vault_server_schema=vault_server_schema)
except (easy_server.ServerFileException, easy_server.VaultFileException) as exc:
    print("Error: {}".format(exc))
    return 1

es_list = esf.list_servers(nickname)  # Works for server and group nicknames

for es in es_list:
    nick = es.nickname

    # The structure of the secrets in the vault file is user-defined.
    # Here, we use the first example vault file.

    host = es.secrets['host'],
    username = es.secrets['username']
    password = es.secrets['password']

    print("Server {n}: host={h}, username={u}, password=********".
          format(n=nick, h=host, u=username))

    # A fictitious session class
    session = MySession(host, username, password)
    . . .

API Reference¶

This section describes the Python API of the easy-server package. The API is kept stable using the compatibility rules defined for semantic versioning. An exception to this rule are fixes for security issues.

Any functions not described in this section are considered internal and may change incompatibly without warning.

ServerFile class¶

class easy_server.ServerFile(filepath, password=None, use_keyring=True, use_prompting=True, verbose=False, user_defined_schema=None, group_user_defined_schema=None, vault_server_schema=None)[source]¶

A server file that specifies the openly accessible portion of the servers and optionally references a vault file that specifies the secret portion of the servers.

An object of this class is tied to a single server file.

The server file is loaded when this object is initialized. If the server file specifies a vault file, the vault file is also loaded at that point.

Optionally, the user-defined portions of the server and group items in the server file, and the server items in the vault file can be validated against user-provided JSON schema.

For a description of the file formats, see sections Server files and Vault files.

Parameters

filepath (unicode string) – Path name of the server file. Relative path names are relative to the current directory.
password (unicode string) – Password for the vault file. None indicates that no password has been provided.
use_keyring (bool) – Enable the use of the keyring service for retrieving and storing the password of the vault file.
use_prompting (bool) – Enable the use of password prompting for getting the password of the vault file.
verbose (bool) – Print additional messages. Note that the password prompt (if needed) is displayed regardless of verbose mode.
user_defined_schema (JSON schema) – JSON schema for validating the values of the user-defined portion of server items when loading the server file. None means no schema validation takes place for these items.
group_user_defined_schema (JSON schema) – JSON schema for validating the values of the user-defined portion of group items when loading the server file. None means no schema validation takes place for these items.
vault_server_schema (JSON schema) – JSON schema for validating the values of the server items when loading the vault file. None means no schema validation takes place for these items.

Raises

ServerFileOpenError – Error opening server file
ServerFileFormatError – Invalid server file format
ServerFileUserDefinedFormatError – Invalid format of user-defined portion of server items in the server file
ServerFileUserDefinedSchemaError – Invalid JSON schema for validating user-defined portion of server items in the server file
ServerFileGroupUserDefinedFormatError – Invalid format of user-defined portion of group items in the server file
ServerFileGroupUserDefinedSchemaError – Invalid JSON schema for validating user-defined portion of group items in the server file
VaultFileOpenError – Error with opening the vault file
VaultFileDecryptError – Error with decrypting the vault file
VaultFileFormatError – Invalid vault file format
VaultFileServerFormatError – Invalid format of server items in the vault file
VaultFileServerSchemaError – Invalid JSON schema for validating server items in the vault file

Attributes:

`filepath`	Absolute path name of the server file.
`vault_file`	Absolute path name of the vault file specified in the server file, or None if no vault file was specified.
`user_defined_schema`	JSON schema for validating the values of the user-defined portion of server items in the server file, or None.
`group_user_defined_schema`	JSON schema for validating the values of the user-defined portion of group items in the server file, or None.
`vault_server_schema`	JSON schema for validating the values of the server items in the vault file, or None.

Methods:

`is_vault_file_encrypted`()	Test whether the vault file is in the encrypted state.
`get_server`(nickname)	Get server for a given server nickname.
`list_servers`(nickname)	List the servers for a given server or server group nickname.
`list_default_servers`()	List the servers for the default server or group.
`list_all_servers`()	List all servers.

property filepath¶

Absolute path name of the server file.

Type: unicode string

property vault_file¶

Absolute path name of the vault file specified in the server file, or None if no vault file was specified.

Vault files specified with a relative path name are relative to the directory of the server file.

Type: unicode string

property user_defined_schema¶

JSON schema for validating the values of the user-defined portion of server items in the server file, or None.

Type: JSON schema

property group_user_defined_schema¶

JSON schema for validating the values of the user-defined portion of group items in the server file, or None.

Type: JSON schema

property vault_server_schema¶

JSON schema for validating the values of the server items in the vault file, or None.

Type: JSON schema

is_vault_file_encrypted()[source]¶

Test whether the vault file is in the encrypted state.

If the server file does not specify a vault file, None is returned.

Returns: Boolean indicating whether the vault file is in the encrypted state, or None if no vault file was specified.
Return type: bool

get_server(nickname)[source]¶

Get server for a given server nickname.

Parameters: nickname (unicode string) – Server nickname.
Returns: Server with the specified nickname.
Return type: Server
Raises: KeyError – Nickname not found

list_servers(nickname)[source]¶

List the servers for a given server or server group nickname.

Parameters: nickname (unicode string) – Server or server group nickname.
Returns: List of servers.
Return type: list of Server
Raises: KeyError – Nickname not found

list_default_servers()[source]¶

List the servers for the default server or group.

An omitted ‘default’ element in the server file results in an empty list.

Returns: List of servers.
Return type: list of Server

list_all_servers()[source]¶

List all servers.

Returns: List of servers.
Return type: list of Server

Server class¶

class easy_server.Server(nickname, server_dict, secrets_dict=None)[source]¶

A data object that represents a single server item from a server file, and optionally the corresponding secrets item from a vault file.

Objects of this class are not created by the user, but are returned by methods of the ServerFile class.

Example for a server item in a server file:

myserver1:                              # nickname of the server
  description: "my dev system 1"
  contact_name: "John Doe"
  access_via: "VPN to dev network"
  user_defined:                         # user-defined part
    stuff: morestuff

Example for a corresponding secrets item in a vault file:

myserver1:                              # nickname of the server
  host: "10.11.12.13"
  username: myuser1
  password: mypass1

Parameters

nickname (unicode string) – Nickname of the server.
server_dict (dict) – Dictionary with the properties of the server item from the server file, wherein optional properties omitted in the server file have been set to their default values.
secrets_dict (dict) – Dictionary with the properties of the secrets item from the vault file, or None if no vault file is specified in the server file or if the vault file does not contain a corresponding item.

Attributes:

`nickname`	Nickname of the server.
`description`	Short description of the server.
`contact_name`	Name of technical contact for the server.
`access_via`	Short reminder on the network/firewall/proxy/vpn used to access the server.
`user_defined`	Additional user-defined properties for the server.
`secrets`	Secrets defined in the vault file for the server, or None if no vault file is specified in the server file or if the vault file does not contain a corresponding item.

property nickname¶

Nickname of the server.

Type: unicode string

property description¶

Short description of the server.

This is the value of the description property of the server item in the server file.

Type: unicode string

property contact_name¶

Name of technical contact for the server.

This is the value of the contact_name property of the server item in the server file. It is optional and defaults to None.

Type: unicode string

property access_via¶

Short reminder on the network/firewall/proxy/vpn used to access the server.

This is the value of the access_via property of the server item in the server file. It is optional and defaults to None.

Type: unicode string

property user_defined¶

Additional user-defined properties for the server.

This is the value of the user_defined property of the server item in the server file. This value can have an arbitrary user-defined structure. It is optional and defaults to None.

Type: dict

property secrets¶

Secrets defined in the vault file for the server, or None if no vault file is specified in the server file or if the vault file does not contain a corresponding item.

Type: dict

VaultFile class¶

class easy_server.VaultFile(filepath, password=None, use_keyring=True, use_prompting=True, verbose=False, server_schema=None)[source]¶

A vault file that specifies the sensitive portion of servers, i.e. the secrets for accessing the servers.

An object of this class is tied to a single vault file.

For a description of the file format, see section Vault files.

The vault file may be in the encrypted or decrypted state. When data from the vault file is read, the vault file remains unchanged, and the data is is decrypted in memory if the file was in the encrypted state.

Vault file encryption, vault file decryption, and reading data from an encrypted vault file requires a password specific to the vault file. The password can be specified as an init argument to this class, or if not provided will be retrieved the keyring service, or if not found there, will be interactively prompted for. The use of the keyring service (for retrieving and storing) and the use of password prompting can be individually disabled.

Typical use in client programs would be to use the keyring and to specify no password (the default). Typical use in test programs running in a CI/CD system would be to specify a password (from the CI/CD system’s secrets) and not to use the keyring.

Parameters

filepath (unicode string) – Path name of the vault file.
password (unicode string) – Password for the vault file. None indicates that no password has been provided.
use_keyring (bool) – Enable the use of the keyring service for retrieving and storing the password.
use_prompting (bool) – Enable the use of password prompting for getting the password.
verbose (bool) – Print additional messages. Note that the password prompt (if needed) is displayed regardless of verbose mode.
server_schema (JSON schema) – JSON schema for validating the values of the server items when loading the vault file. None means no schema validation takes place for these items.

Raises

VaultFileOpenError – Error with opening the vault file
VaultFileDecryptError – Error with decrypting the vault file
VaultFileFormatError – Invalid vault file format
VaultFileServerFormatError – Invalid format of server items in the vault file
VaultFileServerSchemaError – Invalid JSON schema for validating the server items in the vault file

Attributes:

`filepath`	Path name of the vault file.
`nicknames`	Server nicknames in the vault file.
`server_schema`	JSON schema for validating the values of the server items, or None.

Methods:

`is_encrypted`()	Return whether the vault file is encrypted.
`get_secrets`(nickname)	Get the secrets item from the vault file for a given server nickname.

property filepath¶

Path name of the vault file.

Type: unicode string

property nicknames¶

Server nicknames in the vault file.

Type: list of string

property server_schema¶

JSON schema for validating the values of the server items, or None.

Type: JSON schema

is_encrypted()[source]¶

Return whether the vault file is encrypted.

Returns: Boolean indicating whether the vault file is encrypted.
Return type: bool

get_secrets(nickname)[source]¶

Get the secrets item from the vault file for a given server nickname.

Example

Using the following vault file:

secrets:                              # Fixed key
  myserver1:                          # Nickname of the server
    host: "10.11.12.13"               # User-defined secrets
    username: myusername
    password: mypassword

The return value for nickname='myserver1' will be:

dict(
    'host': '10.11.12.13',
    'username': 'myusername,
    'password': 'mypassword',
)

Parameters: nickname (unicode string) – Server nickname.
Returns: Copy of the secrets item for that server from the vault file.
Return type: dict
Raises: KeyError – Nickname not found in the vault file.

Exception classes¶

class easy_server.ServerFileException[source]¶

Abstract base exception for errors related to server files.

Derived from Exception.

class easy_server.ServerFileOpenError[source]¶

Exception indicating that a server file was not found or cannot be accessed due to a permission error.

Derived from ServerFileException.

class easy_server.ServerFileFormatError[source]¶

Exception indicating that an existing server file has some issue with the format of its file content.

Derived from ServerFileException.

class easy_server.ServerFileUserDefinedFormatError[source]¶

Exception indicating that the values of the user-defined portion of server items in a server file do not match the JSON schema defined for them.

Derived from ServerFileException.

class easy_server.ServerFileUserDefinedSchemaError[source]¶

Exception indicating that the JSON schema for validating the values of the user-defined portion of server items in a server file is not a valid JSON schema.

Derived from ServerFileException.

class easy_server.ServerFileGroupUserDefinedFormatError[source]¶

Exception indicating that the values of the user-defined portion of group items in a server file do not match the JSON schema defined for them.

Derived from ServerFileException.

class easy_server.ServerFileGroupUserDefinedSchemaError[source]¶

Exception indicating that the JSON schema for validating the values of the user-defined portion of group items in a server file is not a valid JSON schema.

Derived from ServerFileException.

class easy_server.VaultFileException[source]¶

Abstract base exception for errors related to vault files.

Derived from Exception.

class easy_server.VaultFileOpenError[source]¶

Exception indicating that a vault file was not found or cannot be accessed for reading due to a permission error.

Derived from VaultFileException.

class easy_server.VaultFileDecryptError[source]¶

Exception indicating that an encrypted vault file could not be decrypted.

Derived from VaultFileException.

class easy_server.VaultFileFormatError[source]¶

Exception indicating that an existing vault file has some issue with the format of its file content.

Derived from VaultFileException.

class easy_server.VaultFileServerFormatError[source]¶

Exception indicating that the values of the server items in a vault file do not match the JSON schema defined for them.

Derived from VaultFileException.

class easy_server.VaultFileServerSchemaError[source]¶

Exception indicating that the JSON schema for validating the values of the server items in a vault file is not valid.

Derived from VaultFileException.

Package version¶

easy_server.__version__ = '0.8.0'¶

The full version of this package including any development levels, as a string.

Possible formats for this version string are:

“M.N.P.dev1”: Development level 1 of a not yet released version M.N.P
“M.N.P”: A released version M.N.P

Development¶

This section only needs to be read by developers of the easy-server project, including people who want to make a fix or want to test the project.

Repository¶

The repository for the easy-server project is on GitHub:

https://github.com/andy-maier/easy-server

Setting up the development environment¶

If you have write access to the Git repo of this project, clone it using its SSH link, and switch to its working directory:
```
$ git clone git@github.com:andy-maier/easy-server.git
$ cd easy-server
```
If you do not have write access, create a fork on GitHub and clone the fork in the way shown above.
It is recommended that you set up a virtual Python environment. Have the virtual Python environment active for all remaining steps.
Install the project for development. This will install Python packages into the active Python environment, and OS-level packages:
```
$ make develop
```
This project uses Make to do things in the currently active Python environment. The command:
```
$ make
```
displays a list of valid Make targets and a short description of what each target does.

Building the documentation¶

The ReadTheDocs (RTD) site is used to publish the documentation for the project package at https://easy-server.readthedocs.io/

This page is automatically updated whenever the Git repo for this package changes the branch from which this documentation is built.

In order to build the documentation locally from the Git work directory, execute:

$ make builddoc

The top-level document to open with a web browser will be build_doc/html/docs/index.html.

Testing¶

All of the following make commands run the tests in the currently active Python environment. Depending on how the easy-server package is installed in that Python environment, either the directories in the main repository directory are used, or the installed package. The test case files and any utility functions they use are always used from the tests directory in the main repository directory.

The tests directory has the following subdirectory structure:

tests
 +-- unittest            Unit tests

There are multiple types of tests:

Unit tests

These tests can be run standalone, and the tests validate their results automatically.

They are run by executing:
```
$ make test
```
Test execution can be modified by a number of environment variables, as documented in the make help (execute make help).

An alternative that does not depend on the makefile and thus can be executed from the source distribution archive, is:
```
$ ./setup.py test
```
Options for pytest can be passed using the --pytest-options option.

Contributing¶

Third party contributions to this project are welcome!

In order to contribute, create a Git pull request, considering this:

Test is required.
Each commit should only contain one “logical” change.
A “logical” change should be put into one commit, and not split over multiple commits.
Large new features should be split into stages.
The commit message should not only summarize what you have done, but explain why the change is useful.

What comprises a “logical” change is subject to sound judgement. Sometimes, it makes sense to produce a set of commits for a feature (even if not large). For example, a first commit may introduce a (presumably) compatible API change without exploitation of that feature. With only this commit applied, it should be demonstrable that everything is still working as before. The next commit may be the exploitation of the feature in other components.

For further discussion of good and bad practices regarding commits, see:

Further rules:

The following long-lived branches exist and should be used as targets for pull requests:
- master - for next functional version
- stable_$MN - for fix stream of released version M.N.
We use topic branches for everything!
- Based upon the intended long-lived branch, if no dependencies
- Based upon an earlier topic branch, in case of dependencies
- It is valid to rebase topic branches and force-push them.
We use pull requests to review the branches.
- Use the correct long-lived branch (e.g. master or stable_0.2) as a merge target.
- Review happens as comments on the pull requests.
- At least one approval is required for merging.
GitHub meanwhile offers different ways to merge pull requests. We merge pull requests by rebasing the commit from the pull request.

Releasing a version to PyPI¶

This section describes how to release a version of easy-server to PyPI.

It covers all variants of versions that can be released:

Releasing a new major version (Mnew.0.0) based on the master branch
Releasing a new minor version (M.Nnew.0) based on the master branch
Releasing a new update version (M.N.Unew) based on the stable branch of its minor version

The description assumes that the andy-maier/easy-server Github repo is cloned locally and its upstream repo is assumed to have the Git remote name origin.

Any commands in the following steps are executed in the main directory of your local clone of the andy-maier/easy-server Git repo.

Set shell variables for the version that is being released and the branch it is based on:
- MNU - Full version M.N.U that is being released
- MN - Major and minor version M.N of that full version
- BRANCH - Name of the branch the version that is being released is based on
When releasing a new major version (e.g. 1.0.0) based on the master branch:
```
MNU=1.0.0
MN=1.0
BRANCH=master
```
When releasing a new minor version (e.g. 0.9.0) based on the master branch:
```
MNU=0.9.0
MN=0.9
BRANCH=master
```
When releasing a new update version (e.g. 0.8.1) based on the stable branch of its minor version:
```
MNU=0.8.1
MN=0.8
BRANCH=stable_${MN}
```

Create a topic branch for the version that is being released:

git checkout ${BRANCH}
git pull
git checkout -b release_${MNU}

Edit the version file:
```
vi easy_server/_version.py
```
and set the __version__ variable to the version that is being released:
```
__version__ = 'M.N.U'
```
Edit the change log:
```
vi docs/changes.rst
```
and make the following changes in the section of the version that is being released:
- Finalize the version.
- Change the release date to today’s date.
- Make sure that all changes are described.
- Make sure the items shown in the change log are relevant for and understandable by users.
- In the “Known issues” list item, remove the link to the issue tracker and add text for any known issues you want users to know about.
- Remove all empty list items.
When releasing based on the master branch, edit the GitHub workflow file test.yml:
```
vi .github/workflows/test.yml
```
and in the on section, increase the version of the stable_* branch to the new stable branch stable_M.N created earlier:
```
on:
  schedule:
    . . .
  push:
    branches: [ master, stable_M.N ]
  pull_request:
    branches: [ master, stable_M.N ]
```

Commit your changes and push the topic branch to the remote repo:

git status  # Double check the changed files
git commit -asm "Release ${MNU}"
git push --set-upstream origin release_${MNU}

On GitHub, create a Pull Request for branch release_M.N.U. This will trigger the CI runs.

Important: When creating Pull Requests, GitHub by default targets the master branch. When releasing based on a stable branch, you need to change the target branch of the Pull Request to stable_M.N.
On GitHub, close milestone M.N.U.
On GitHub, once the checks for the Pull Request for branch start_M.N.U have succeeded, merge the Pull Request (no review is needed). This automatically deletes the branch on GitHub.
Add a new tag for the version that is being released and push it to the remote repo. Clean up the local repo:
```
git checkout ${BRANCH}
git pull
git tag -f ${MNU}
git push -f --tags
git branch -d release_${MNU}
```
When releasing based on the master branch, create and push a new stable branch for the same minor version:
```
git checkout -b stable_${MN}
git push --set-upstream origin stable_${MN}
git checkout ${BRANCH}
```
Note that no GitHub Pull Request is created for any stable_* branch.
On GitHub, edit the new tag M.N.U, and create a release description on it. This will cause it to appear in the Release tab.

You can see the tags in GitHub via Code -> Releases -> Tags.
On ReadTheDocs, activate the new version M.N.U:
- Go to https://readthedocs.org/projects/easy-server/versions/ and log in.
- Activate the new version M.N.U.
  
  This triggers a build of that version. Verify that the build succeeds and that new version is shown in the version selection popup at https://easy-server.readthedocs.io/
Upload the package to PyPI:
```
make upload
```
This will show the package version and will ask for confirmation.

Attention! This only works once for each version. You cannot release the same version twice to PyPI.

Verify that the released version arrived on PyPI at https://pypi.python.org/pypi/easy-server/

Starting a new version¶

This section shows the steps for starting development of a new version of the easy-server project in its Git repo.

This section covers all variants of new versions:

Starting a new major version (Mnew.0.0) based on the master branch
Starting a new minor version (M.Nnew.0) based on the master branch
Starting a new update version (M.N.Unew) based on the stable branch of its minor version

The description assumes that the andy-maier/easy-server Github repo is cloned locally and its upstream repo is assumed to have the Git remote name origin.

Any commands in the following steps are executed in the main directory of your local clone of the andy-maier/easy-server Git repo.

Set shell variables for the version that is being started and the branch it is based on:
- MNU - Full version M.N.U that is being started
- MN - Major and minor version M.N of that full version
- BRANCH - Name of the branch the version that is being started is based on
When starting a new major version (e.g. 1.0.0) based on the master branch:
```
MNU=1.0.0
MN=1.0
BRANCH=master
```
When starting a new minor version (e.g. 0.9.0) based on the master branch:
```
MNU=0.9.0
MN=0.9
BRANCH=master
```
When starting a new minor version (e.g. 0.8.1) based on the stable branch of its minor version:
```
MNU=0.8.1
MN=0.8
BRANCH=stable_${MN}
```

Create a topic branch for the version that is being started:

git checkout ${BRANCH}
git pull
git checkout -b start_${MNU}

Edit the version file:
```
vi easy_server/_version.py
```
and update the version to a draft version of the version that is being started:
```
__version__ = 'M.N.U.dev1'
```

Edit the change log:

vi docs/changes.rst

and insert the following section before the top-most section:

Version M.N.U.dev1
------------------

This version contains all fixes up to version M.N-1.x.

Released: not yet

**Incompatible changes:**

**Deprecations:**

**Bug fixes:**

**Enhancements:**

**Cleanup:**

**Known issues:**

* See `list of open issues`_.

.. _`list of open issues`: https://github.com/andy-maier/easy-server/issues

Commit your changes and push them to the remote repo:

git status  # Double check the changed files
git commit -asm "Start ${MNU}"
git push --set-upstream origin start_${MNU}

On GitHub, create a Pull Request for branch start_M.N.U.

Important: When creating Pull Requests, GitHub by default targets the master branch. When starting a version based on a stable branch, you need to change the target branch of the Pull Request to stable_M.N.
On GitHub, create a milestone for the new version M.N.U.

You can create a milestone in GitHub via Issues -> Milestones -> New Milestone.
On GitHub, go through all open issues and pull requests that still have milestones for previous releases set, and either set them to the new milestone, or to have no milestone.
On GitHub, once the checks for the Pull Request for branch start_M.N.U have succeeded, merge the Pull Request (no review is needed). This automatically deletes the branch on GitHub.

Update and clean up the local repo:

git checkout ${BRANCH}
git pull
git branch -d start_${MNU}

Appendix¶

Glossary¶

string¶: a unicode string or a byte string
unicode string¶: a Unicode string type (unicode in Python 2, and str in Python 3)
byte string¶: a byte string type (str in Python 2, and bytes in Python 3). Unless otherwise indicated, byte strings in this project are always UTF-8 encoded.
JSON schema¶: a dict object that represents a JSON schema (i.e. as if it had been loaded from a JSON schema file using json.load()). For details about how to define a JSON schema, see https://json-schema.org/understanding-json-schema/.

References¶

Python glossary¶

Change log¶

Version 0.8.0¶

This version contains all fixes up to version 0.7.1.

Released: 2021-05-01

Bug fixes:

Fixed the increase of the development status to Beta. (issue #29)

Enhancements:

Added support for JSON schema-validation of user-defined portions of server and group items in server file and of server items in vault file.

Cleanup:

Docs: Removed unused ‘vault_file’ variable from example code.

Version 0.7.0¶

Released: 2021-04-05

Incompatible changes:

The ‘VaultFile’ class now raises ‘VaultFileOpenError’ for vault file not found. Previously, it raised ‘easy_vault.EasyVaultFileError’ (related to issue #10)

Bug fixes:

Added missing exception transformation to ‘VaultFileOpenError’ for vault file not found in ‘VaultFile’ class. (related to issue #10)

Enhancements:

Docs: Updated usage example and example scripts to use integrated vault. (issue #26)
In ‘VaultFile’ class, the input ‘filepath’ is now made absolute. (related to issue #10)
Added a method ‘VaultFile.is_encrypted()’ to return whether the vault file, is encrypted.
Added a method ‘ServerFile.is_vault_file_encrypted()’ to return whether the vault file of the server file (if any) is encrypted.
Improved text coverage of ‘VaultFile’ class. (issue #10)
Increased minimum version of easy-vault package to 0.7.0. (issue #40)
Increased development status to Beta. (issue #29)

Version 0.6.0¶

Released: 2021-04-02

Incompatible changes:

The new optional ‘use_prompting’ parameter of the ‘VaultFile’ and ‘ServerDefinitionFile’ classes was not added at the end of the parameter list. This is incompatible for users who called the function with positional arguments. (related to issue #22)
Renamed the following classes for simplicity: ‘ServerDefinitionFile’ to ‘ServerFile’, ‘ServerDefinition’ to ‘Server’, ‘ServerDefinition…’ exceptions to ‘Server…’

Enhancements:

Integrated vault access into server definition file. The server definition files can now optionally specify the path name of a vault file. If specified, the vault file is loaded as well and the secrets for a server defined in the vault file are available in the ServerDefinition object as a new secrets property. (issue #20)
In the ‘VaultFile’ and ‘ServerDefinitionFile’ classes, added a new parameter ‘use_prompting’ that allows disabling the interactive prompting for passwords. Also, changed the logic for requiring passwords such that they are only required when the vault file is being encrypted, decrypted or accessed in the encrypted state. (issue #22)

Version 0.5.0¶

Released: 2021-03-29

Initial release.