conformance/Fixture

What a tool source gives the conformance suites. The suites need a system to run the tools of: an MCP server, an HTTP API, whatever the source reaches. The fixture serves three tools of it, which every source is tested with, and makes the source for one test over a system in the condition the suite asks for.

Since v0.1.0

Testing

Condition (type alias)

How the system the source reaches is for a test: it works, it refuses the credentials the source has, it answers that it is down, or it cannot be reached.

Signature

type Condition = "working" | "unauthorized" | "unavailable" | "unreachable"

Source

Since v0.1.0

Fixture (interface)

What a source’s tests give run: how to make the source over a system in each condition, and the three tools of that system the suites call, echo, failing, and blocking.

Signature

export interface Fixture<E> {
  /**
   * Makes the source for one test, over a system in `condition`. What it starts ends with the
   * scope of the test.
   */
  readonly make: (condition: Condition) => Effect.Effect<Subject, E, Scope.Scope>

  /** The system's tools that the suites call, each by the name the source discovers it under. */
  readonly tools: {
    /** Answers with the text it is given, which is somewhere in what the source returns. */
    readonly echo: {
      readonly tool: string

      /** The input that asks the tool to answer with `text`. */
      readonly input: (text: string) => Schema.JsonObject

      /** An input that the tool's parameters do not allow. */
      readonly invalidInput: Schema.JsonObject
    }

    /** Reports that it failed, whatever its input. */
    readonly failing: ToolCase

    /** Does not answer, until the call that asked it is given up. */
    readonly blocking: ToolCase
  }

  /** How many milliseconds a test may take, more than the default for a source on the network. */
  readonly timeout?: number
}

Source

Since v0.1.0

Subject (interface)

The source under test, for one test, with what the suites need of the system behind it.

Signature

export interface Subject {
  readonly source: ToolSource.ToolSource

  /** Completes once the system has been asked to run the blocking tool. */
  readonly blockedCallStarted: Effect.Effect<void>

  /**
   * What the source holds that must not reach a model, which is credentials, and the address of the
   * system if it could say something of what is inside.
   */
  readonly secrets: ReadonlyArray<string>
}

Source

Since v0.1.0

ToolCase (interface)

A tool of the fixture, and the input that calls it.

Signature

export interface ToolCase {
  /** The name the source discovers the tool under, short enough to stay as it is in an assembly. */
  readonly tool: string

  /** An input that the tool's parameters allow. */
  readonly input: Schema.JsonObject
}

Source

Since v0.1.0