Made of little things.

# soyLI tooling, skills and bootstrapper report

> Follow-up: [responsive editing and soyLI 0.23.1 verification](responsive-editing.md).
> The original findings below describe the earlier 0.23.0 run.

Observed while building Napcraft on 2026-09-25, macOS arm64, soyLI 0.23.0.
The workspace was already scaffolded and dependencies/skills were already installed.
I did not exercise `soyli new` or a clean first-time installer, so this is an audit
of the generated workspace and subsequent authoring flow, not a complete installer test.

## Executive assessment

The runtime and dynamic backend are sufficient for this creation. Persistent
worlds, verified owner commands, guest edits, idempotent retries, conflict handling
and actual two-player WebRTC worked. The principal release blocker is a packaging
omission in the temporary projects used by check/capture/multiplayer. The guidance
is strong on authority boundaries and honest testing, but its bootstrap/configuration
story needs consolidation and authenticated multiplayer needs a supported fixture.

| Priority | Finding | Confidence |
| --- | --- | --- |
| P0 | Temporary check/capture/multiplayer projects omit declared module files | Reproduced; installed code inspected |
| P1 | Public backend-context guidance conflicts with source-selection rejection | Reproduced |
| P1 | `backend init` reports napplet.json while writing local binding overrides | Observed and binding inspected |
| P1 | Guest-only multiplayer starter cannot create an account-required world | Contract gap; custom host fixture needed |
| P2 | Managed toolchain operations are awkward under an agent filesystem sandbox | Reproduced environment interaction |
| P2 | Browser/test executable discovery and TURN compatibility need a supported path | Observed integration friction |
| P2 | Adapted/upstream skill routing repeats contradictory scaffolding instructions | Read directly in installed guidance |
| P2 | Backend context types and conformance evidence need sharper examples | Encountered during implementation |

## 1. Release-blocking temporary-project omission

`soyli backend check` compiles the module and `soyli dev` runs it correctly.
Both `soyli check` and `soyli multiplayer` fail before useful browser work with:

```text
BROWSER_CHECK … ENOENT … /napplet-publish-check-…/backend
ENOENT … /soy-multiplayer-…/backend
```

The installed check path writes the artifact, napplet.json and managed assets into
its temporary project, then initializes the local backend. It never writes the
declared module manifest, handler and schemas there. The multiplayer runner has
the same omission. Tracking/committing the module files does not fix it. Recording
and screenshots share the frozen check path, so the defect affects finishing as
well as testing/publication.

Fix: populate the isolated project with validated module files from the frozen
source snapshot before backend startup. Do not substitute current working-tree
bytes after freezing. Test nested paths, multiple modules, invalid references,
cleanup on failure and isolation from existing dev state. Keep source path and
schema/security validation intact. See the smaller reproduction and regression
proposal in [soyli-backend-capture-bug.md](soyli-backend-capture-bug.md).

`soyli update` confirmed 0.23.0 was current. No bypass, release or public backend
deployment was performed. The creator said a patch will be implemented next.

## 2. Public context versus private-directory policy

The backend guide instructs app code to import
`.napplet-space/soy-backend.json` and says to keep it in the source release so other
creators can inspect destinations. The bootstrapper ignores `.napplet-space/`.
Force-tracking that one generated public context file caused `soyli check` to fail
with `SOURCE_SECRET`. Removing it from the Git index allowed source validation to
continue to the separate temporary-backend bug. The file contains public provider,
napplet address and module names, not a signer secret.

Choose one explicit supported contract:

- Export this public context to a tracked path outside the private binding directory;
- narrowly permit this generated public file in source packaging; or
- guarantee regeneration from portable metadata during setup/build/remix, and stop
  telling authors to retain the generated file in the source release.

Do not broadly allow the rest of `.napplet-space/` into publication. Add a fresh
checkout/remix build test that proves the import exists without copying private
bindings, local SQLite state or publication journals.

## 3. `backend init` configuration provenance is confusing

The command reported `config: "napplet.json"` and successfully generated the public
context. The portable napplet.json still had no `backend` property. Inspection
confirmed the provider and empty boards were instead under
`.napplet-space/project.json` → `project.backend`. This is an actual local override,
not lost data or a failed initializer.

The distinction matters because the docs then show adding `backend.modules` to
ordinary napplet.json, and the app must remain reproducible in another checkout.
I made the module/provider declaration explicit in the portable file.

Recommendation: report the actual written path and whether each value is portable
or a local override. A machine-readable effective config should preserve field
provenance. Provide one canonical initialized-module example including what is
tracked, what is regenerated, and how remix preserves the backend declaration.

## 4. Authenticated multiplayer and persistent-world test fixtures

The supplied scenario API is useful: independent contexts, assertions, timing and
transport diagnostics are the right primitives. However, its documented default
players are guests, while dynamic instance creation always requires a verified
account. The example only supplies Host/Join selectors and does not show how to
establish test identities and authorize account-bound operations.

I needed a trusted host-side NIP-07 fixture using ephemeral generated identities,
real signed authorization events, the actual host Connect button and its consent
prompts. The app still used the normal SDK and never received keys. A second
context remained a guest. This proved actual ownership checks instead of weakening
creation to accommodate a test.

Provide a supported `testAccount`/`connectIdentity` fixture that is available only
to local tests, plus helpers for consenting to the expected provider/account and
peer connection. Include a dynamic-world scenario covering owner creation, guest
join, unauthorized owner actions, simultaneous edits, exact committed-request
retry, everyone leaving, backend restart and release pinning. Also expose test
fixtures to captures so a shared-world preview does not require ad hoc automation.

The existing `backend check` correctly says compilation is not game-rule proof.
That distinction should stay.

## 5. Managed toolchain and agent sandbox

`soyli run type-check` failed in the agent sandbox with `EPERM` while attempting to
chmod an existing cached pnpm entry under `~/Library/Caches/napplet-space/toolchains`.
Plain `pnpm type-check`, `pnpm build` and guidance tests worked. An attempted offline
`pnpm add -D` then failed with `ERR_PNPM_UNEXPECTED_STORE`: bootstrap dependencies
were linked from the managed soyLI store, while ordinary pnpm selected a different
store. No extra runtime dependency was ultimately needed or added.

This is not evidence that pnpm/bootstrap installation is broken outside a sandbox.
It is friction between toolchain ownership and the promised ordinary-pnpm workflow.
Avoid rewriting permissions on an already usable toolchain during every run, and
provide/document the managed dependency-add command and effective store path. A
read-only toolchain cache should remain sufficient for running already-installed
project tools. Keep managed binaries pinned.

Initial public-provider discovery also failed under restricted networking, then
succeeded after normal execution escalation. That was an environment permission
boundary, not a napplet.soy outage. The structured diagnostic and recovery text
were helpful.

## 6. Browser discovery, local ICE and coturn

The general browser tool expected Google Chrome under `/Applications`, where it
was absent. soyLI and the project already had usable cached Chromium/Playwright.
I used that installed browser through the project's pinned packages. A supported
`soyli browser path --json` (or equivalent doctor output) would remove hardcoded
cache paths and dependency-directory discovery from author tests.

Direct local ICE negotiated descriptions but stalled in `checking` here. That
alone does not establish a soyLI signaling bug. An authenticated disposable
loopback coturn relay over TCP worked with the real host, two separate contexts
and 50 ms ±15 ms added outgoing delay. Guest input-to-render was 12.9 ms; remote
avatars, edits and rejoining passed. The local relay says nothing about deployed
TURN or independent public networks.

The first coturn invocation I assembled used an obsolete `--no-dtls` option, which
coturn 4.18 rejected. This was test-setup friction, not a Napcraft defect; it is
not proof that soyLI's config-file helper fails. Pin or qualify a supported coturn
version and feature-detect incompatible options. A built-in disposable relay test
would substantially improve reliability on VPN/restricted-network machines.

The fallback test harness starts and stops its own relay. No always-on service was
installed or enabled. The coturn executable remains available for future tests.

## 7. Skills and workspace guidance

What worked well:

- `docs/napplet-space.md` clearly overrode upstream scaffold commands for this
  already-created soyLI workspace and correctly directed preview through the host.
- The make → design → UI → build → test flow prevented accidental shell code,
  direct networking, browser persistence or app-owned signing.
- The soy-backends skill correctly separated durable rules from fast movement,
  required exact retry identities, and emphasized pinned releases and restart tests.
- The visual/mobile guidance led to a distinctive app-owned palette, actual frame
  dimension assertions, touch testing and honest physical-device limitations.
- The warnings about the host's 300px iframe minimum were directly useful: the
  first nominal 200×160 resize still measured 200×300 until the host override and
  asynchronous viewport update were handled. The test then asserted exact sizes.
- The capture guidance required inspecting real media, not just successful file
  creation. The final PNG and full seven-second clip were verified in Listing.

What should change:

- The adapted `napplet-build` skill still contains unconditional upstream
  `napplet create`/`napplet init` instructions. Workspace overrides resolve this,
  but the entry point should branch explicitly on pre-scaffolded soyLI state.
- Several skills repeat long protocol and scaffold sections. Put authoritative
  tooling adaptations in one concise shared reference and link it consistently.
  State clearly which existing instructions are superseded.
- The generic build skill's small edit-surface table underspecifies legitimate
  backend modules, test scenarios and split rendering/network code. Keep the
  substrate protected, but explicitly allow those product files.
- The skills should distinguish "locally complete, release blocked by tooling"
  from "done" with a compact evidence template and a concrete patch handoff.
  The current insistence on not claiming skipped/blocked work as passing is good.

## 8. Types, errors and evidence quality

Document the representation of `ctx.owner` explicitly. It is a principal string
such as `nostr:<pubkey>`, whereas `ctx.account` is the verified raw key. My initial
extra owner guard compared them incorrectly; the real-host test caught it. That
was an application bug made easier by an underspecified example type, not a
provider authorization failure. Use a shared typed Context contract and an owner
comparison example, or show when schema `access: "owner"` already performs the
check.

Likewise, my initial presence sender tried to send before any peer had joined.
The fix was to wait for the real peer event. This is app lifecycle behavior, not a
transport defect. A complete sample should include that state transition.

`backend status --json` was rechecked with stdout/stderr separated: stdout is one
valid JSON document. Transient relay retry logs go to stderr. Do not file a JSON
format bug based on a terminal's combined output. The `error: {}` / level-50
"Publish failed; will retry" messages alongside an overall successful status are
still hard to interpret; distinguish recovered retries from final failures.

The reference conformance suite reported five passes and five skips. It did not
measure manifest events, actual wire envelopes or lifecycle for this artifact.
A prominently structured evidence summary would prevent callers treating
"CONFORMANT" as proof of all ten checks or of gameplay. The workspace documentation
already warns about this; preserve that honesty and improve the output contract.

## Suggested implementation order

1. Fix frozen temporary-project backend copying; regress check, screenshot,
   recording, multiplayer and cleanup.
2. Resolve portable backend metadata versus private binding/context generation.
3. Add authenticated local test accounts and dynamic-world examples.
4. Make toolchain/cache/browser discovery friendly to sandboxed agents.
5. Consolidate adapted skills, strengthen Context types and publish a structured
   verification report with passed/failed/skipped/blocked states.

Napcraft is committed locally with working source, tests, screenshots and a real
multiplayer clip. Publication and public-provider admission remain unverified until
the P0 fix is shipped; the original creator identity and publishing destinations
have been preserved.