Custom validators and formats

When format rules are not enough, a property can be checked by a Python function. It runs in Pydantic Monty, a sandboxed Python interpreter written in Rust, and a write is accepted only if every call passes.

Writing one

A validator is Python code that defines validate(value):

def validate(value):
    # value is the property's normalized JSON value: str, int, float, bool, list, dict
    if not value.startswith("PRJ-"):
        return "project codes start with PRJ-"
    digits = value[4:]
    if not digits.isdigit() or len(digits) != 4:
        return False
    return True

It passes by returning True or None, and fails by returning False (reported as “rejected by the validator”), by returning a string (the failure message, cut at 300 characters), or by raising (the exception is the message). Returning anything else fails too. Anything else that goes wrong also fails the value: running out of time or memory, a crash, or touching anything outside the sandbox.

Validators belong to the organization and are managed with manage_ontology:

POST /v1/ontology/validators
{ "name": "project_code", "description": "PRJ- and four digits", "code": "def validate(value): ..." }

The server first checks, in the sandbox, that the code defines validate. Each POST with an existing name adds the next version. Versions are immutable: the organization API’s database role may only insert and read them. Code is limited to 20,000 characters.

POST /v1/ontology/validators/{name}/test runs a version (the latest by default) on up to 100 sample values without writing anything:

{ "name": "project_code", "version": 2, "results": [
  { "valid": true },
  { "valid": false, "message": "project codes start with PRJ-" }
] }

Using one

A property refers to a validator with a rule that pins one version:

{ "api_name": "code", "type": { "kind": "string" }, "required": true,
  "rules": [{ "rule": "validator", "name": "project_code", "version": 2 }] }

The rule works on any property type and on nested values (array elements, struct fields). Because the rule pins a version, publishing version 3 changes nothing until a PUT of the object type uses it. That PUT, like any rules change, applies to new writes at once and to stored objects in the background (see Ontology), validators included.

When they run

On every object write, after the declarative rules pass, the server runs every validator call the object’s values owe, grouped as one sandbox run per validator. The write is accepted only if all of them pass. Failures appear in the 422 validation_failed details with the path of the value.

Time limits:

• object writes: all calls within 5 s, and at most 1000 calls per object; a write that runs out of time is refused with 422,
• the background job that checks stored objects after a rules change: a batch of objects within 60 s; if the batch runs out of time, each object is retried alone within 5 s, and an object that still runs out is marked invalid.

If no sandbox is configured (MONTY_WORKER=off, or no worker binary next to the server), writes that owe validator calls get 503 validators_unavailable; writes that owe none are unaffected. Stored objects of a type whose rules owe calls stay pending until a sandbox is available.

Formats

A format is a validator for strings. The four built-in formats, email, url, uuid and cuid, are checked natively in Rust; an organization adds its own in Python:

POST /v1/ontology/formats
{ "name": "sku", "description": "three capitals, a dash, digits", "code": "def validate(value): ..." }

{ "api_name": "sku", "type": { "kind": "string" },
  "rules": [{ "rule": "format", "name": "sku", "version": 1 }] }

Formats are written, versioned, tested (POST /v1/ontology/formats/{name}/test) and run exactly like validators, with two differences: the rule fits string properties only, and formats have their own names, so a format and a validator may both be called sku. Failures read format sku v1: <message>. GET /v1/ontology/formats lists the built-ins next to the organization’s formats, and their names cannot be taken.

The built-ins stay native on purpose. They are fixed, total and fast, so they never cost a sandbox run, never time out, and keep working when no sandbox is configured: an email property never gets a 503. Moving them into Monty would add a subprocess round trip to most writes and make them fail with the sandbox, for no gain in flexibility, since an organization that wants a different notion of an email address can write its own format.

The sandbox

Monty is a Python interpreter written in Rust. It offers no host functions, filesystem, environment, network, or imports beyond its built-in modules: a name the code does not define is a NameError, and any OS call (files, environment, clock, sleep) ends the run as a failure.

Results depend only on the code and the value: the clock is fixed at the moment the run starts and random is seeded per run.

Runs happen in worker subprocesses (happy-enterprise-monty-worker, deploy it next to the server binary) with an empty environment and a memory-limited allocator. Each worker serves one run (one request’s calls to one validator) and is then replaced, so nothing a validator does survives into another run or reaches another organization.