Testing and validation

Two layers keep a package honest. At load time, the host validates the manifest and every contribution before anything registers — a package that fails validation does not load. At development time, @neuralis/package-system/testing provides the shared helpers every first-party suite uses, so your tests exercise the same contract shapes the runtime dispatches.

Load-time validationCopy Anchor Link

Validation results are structured ({ valid, errors, warnings }) — errors block registration, warnings surface in logs.

The testing toolkitCopy Anchor Link

Everything below imports from @neuralis/package-system/testing and runs under vitest.

Session factoriesCopy Anchor Link

All four accept partial overrides, so boundary tests pin exactly the fields they care about (see Sessions for the shape).

Request and context buildersCopy Anchor Link

Route helpers build the PackageRouteRequest your handler receives — routeGet(path, query?, session?), routePost(path, body?, session?), routePatch(path, body?, session?), routeDelete(path, session?) (path is the segment array after /api/packages/<id>/), plus the generic createMockRouteRequest(overrides?). You then call the handler directly:

import { describe, it, expect } from 'vitest';
import { routeGet, createDeniedSession } from '@neuralis/package-system/testing';
import { GET } from '../src/routes/items';

describe('items route', () => {
  const state = { db: { listItems: async () => [{ id: '1' }] } };

  it('returns items for an authorized caller', async () => {
    const res = await GET(routeGet(['items'], undefined, { grantedFeatures: ['my-pkg.read'] }), state);
    expect(res.status).toBe(200);
  });

  it('denies a caller without the feature', async () => {
    const res = await GET(routeGet(['items'], undefined, createDeniedSession()), state);
    expect(res.status).toBe(403);
  });
});

For tools and lifecycle code:

• createMockToolCallContext() / createDeniedToolCallContext() — the context a tool handler receives (session + init context), with sessionOverrides / ctxOverrides knobs.
• createMockInitContext({ packageSlug?, dataDir?, globalDir?, ... }) — a first-party context by default (globalDir: null builds a project-installed package's context) — a PackageInitContext with a real FileStore/JsonlAppender and a working dataPath wired into a temp directory, so a test exercises the real scoping rather than a permissive double.
• createMockPackagesAccessor({ '@scope/pkg': api }) — cross-package API stubbing for getPackageApi() consumers.
• createDiscoveredRoute / createDiscoveredToolHandler — dispatcher test subjects for exercising RouteDispatcher / ToolDispatcher behavior.
• createPackageDefinition / createToolDefinition plus the hosted-entry builders — minimal valid contract shapes with overrides.

Filesystem and async helpersCopy Anchor Link

• createTempDir(prefix) / cleanupTempDirs() / withTempDir(prefix, fn) — auto-cleaned scratch directories for filesystem-touching tests.
• waitFor(predicate, { timeoutMs?, intervalMs?, message? }) — flake-safe condition polling; throws on timeout (default 5 s).

Connector complianceCopy Anchor Link

import { assertConnectorCompliance } from '@neuralis/package-system/testing';

await assertConnectorCompliance(connector, {
  rootUri: `${connector.source}://`,
  sampleFilePath: `${connector.source}://README.md`,
  packageTrust: 'trusted',
});

The harness exercises every capability the connector declares — including the declared-versus-actual capability match, write/read/delete round-trips (only when advertised), and the typed error taxonomy — and throws a ConnectorComplianceFailure listing every violation. It never writes outside the rootUri you provide. Strongly recommended for every source connector.

Cross-package mirror driftCopy Anchor Link

Where a dependency edge exists, a cross-package call site simply imports the provider's canonical type and tsc does the work. Where it must not — a cycle, a package that deliberately does not depend on the provider, a generic host — the call site declares a small structural shape instead, and nothing holds the two together. assertApiMirror closes that by compiling the mirror against the file that DECLARES the provider type, addressed by absolute path so no dependency edge is created:

import { assertApiMirror } from '@neuralis/package-system/testing';

expect(assertApiMirror({
  mirrorFile: `${repo}/packages/your-package/src/theirApi.ts`,
  mirrorType: 'TheirApiLike',
  providerDts: `${repo}/packages/their-package/dist/src/lifecycle.d.ts`,
  providerType: 'TheirApi',
})).toEqual([]);

It reports both directions of drift — a key the provider does not have (the diagnostic names it) and a changed argument or return shape — and treats vacuity as a failure, never a skip: a missing declaration file, or any unresolved first-party module in the program, is reported, because an unresolved provider collapses to any and both checks would pass for free. It reads the declaration as last emitted, so build the provider before the suite that pins it. Pass providerType: '*' to compile against the module shape a dynamic import('<pkg>') hands back, and exclude for a key the call site owns itself.

Feature auditCopy Anchor Link

auditFeatures({ provided, references }) is the analysis core for keeping feature wiring clean: it reports ids that are referenced (by a tool, route, skill, or role grant) but that no package provides, and declared features nothing references. formatFeatureAudit(result) renders the findings for logs. Feed it the ids from your manifests and the references your suite collects.

What to coverCopy Anchor Link

Test the contract, not just the happy path:

• Authorization — every route and tool against a denied session; verify 403/error results, not exceptions.
• Feature gates — each declared feature with and without the grant, plus the '*' wildcard (see Features and access).
• Cross-project isolation — handlers must scope queries by req.session.projectId; assert that data from another project never leaks.
• Schema strictness — run validateToolSchema over your tools/*.json in a test so a drifting schema fails CI, not production load.
• Failure paths — malformed input, oversized payloads, missing state, connector errors; assert structured error results.
• Result envelopes — runtime-only accounting belongs in _meta, never in model-visible content.

App surface pre-flightCopy Anchor Link

A App surface is the contribution whose failures are almost all silent: it validates, it loads, the snapshot lists it, and the panel is blank or unstyled. Load-time validation covers only what the manifest object can express, so the rest is a pre-flight and a look at the rendered surface.

Which subresource rules apply depends on the entry's asset mode, and neither set can be checked at load time: the validator receives the manifest object with no package root and no filesystem access, so it cannot inspect a file's content. A self-contained entry's linked stylesheet is simply never served — the request leaves an opaque origin carrying no session cookie and nothing answers it. A bundle entry's siblings are served, from a lane that refuses anything resolving outside the surface root and app/shared/. The pre-flight script ships with the package-creation skill; run it before every publish.