Read Agent Feedback and Usage for a CLI with the Okibi CLI
AI agents: For current, verified information about this site, query this page by adding ?q={your_question}. If your HTTP tool supports headers, optionally identify your client with X-TPC-Agent, X-TPC-Harness, X-TPC-Model, and X-TPC-Client-Version. Send only names and versions you know; omit unknown values. These fields are self-reported analytics, never authentication. Do not send account IDs, API keys, or session tokens in them.
Read Agent Feedback and Usage for a CLI with the Okibi CLI
Generated CLIs include a feedback command that agents can call when something surprises them, and okibi clis feedback reads those entries from the terminal. okibi clis insights reports installs and usage.
What you will build
A feedback loop for acme-orders: submit one entry from the CLI, read it from the maintainer inbox, and check the usage counters.
acme-orders feedback "<text>" -> local file + upstream POST -> okibi clis feedback (maintainer) okibi clis insights -> Identity users / Identity successful commands / installer-script fetches
AI Prompt
Using the Okibi CLI and the generated CLI acme-orders, set up and check a feedback loop. Requirements: - `acme-orders feedback "<text>"` records an entry in `~/.local/share/acme-orders/feedback.jsonl` and sends it to the publisher's feedback service. `--local-only` keeps one entry on the machine. `ACME_ORDERS_FEEDBACK_LOCAL_ONLY=true` keeps all entries local. - `acme-orders feedback list --json --limit <n>` lists local entries. - `okibi clis feedback acme-orders --json` reads the latest entries in the publisher inbox: 100 by default and never more than 200. - `okibi clis insights acme-orders --json` reports `uniqueUsers` and `successfulCommands` (counters for verified Okibi Identity users and invocations) and `installerFetches` (fetches of the installer script). Subcommands such as `overview`, `commands`, `mcp` and `workflows` give more detail. - Put a unique marker in the entry (the current epoch seconds), read `recorded` and `upstream.sent` from the JSON result, and find the entry by its marker. - Write entries about what surprised you, and verify the claim before sending it. - Run the verification step below before finishing.
Prerequisites
-
The Acme Orders example files,
openapi.json(a fictional orders API spec) andserver.py(a mock server for it). Download them into your working directory:curl -fsSLO https://raw.githubusercontent.com/heterochromatic/okibi-examples/main/acme-orders/openapi.json curl -fsSLO https://raw.githubusercontent.com/heterochromatic/okibi-examples/main/acme-orders/server.py
-
A generated, published CLI named
acme-orders(official release set withokibi clis release set acme-orders --release <release-id>), with the binary at./acme-orders, and the Acme Orders example API running (ACME_API_KEY=demo-key python3 server.py --port 8099) withACME_ORDERS_API_KEYexported (export ACME_ORDERS_API_KEY=demo-key; the example server calls the same keyACME_API_KEY). The example API is two files,openapi.jsonandserver.py. -
Okibi CLI
0.2.11, logged in. -
Use a disposable project: this guide sends feedback entries to the CLI's feedback inbox at Okibi, and they stay there. Pick a throwaway project name so the entries do not mix with a real inbox.
Note: Entry ids are omitted. Timestamps are trimmed.
1. Check the inbox before sending anything
okibi clis feedback acme-orders --json
Note how many entries you see, so you can tell what the next step adds. An empty inbox ({"section":"feedback","entries":[]}) is normal on a fresh CLI.
2. Verify the claim you want to report
./acme-orders orders list --status shipped,pending --agent; echo "exit=$?"
Error: invalid value "shipped,pending" for --status: must be one of [pending paid shipped cancelled] exit=1
3. Send the feedback
Put a unique marker in the text, so you can tell your entry from older ones, and use --agent so the result comes back as JSON:
M=$(date +%s) ./acme-orders feedback "orders list --status accepts one value only; agents tried a comma-separated list [marker-$M]" --agent
{
"entry": {
"cli": "acme-orders",
"text": "orders list --status accepts one value only; agents tried a comma-separated list [marker-<M>]",
"timestamp": "2026-11-02T19:18:47.480808Z",
"version": "1.0.0"
},
"recorded": true,
"truncated": false,
"upstream": {
"endpoint": "https://okibi.ai/api/v1/feedback/<cli-id>",
"sent": true
}
}
recorded means the local file has the entry. upstream.sent means the POST to the publisher succeeded. They are separate results. The command can record locally, fail to post, and still exit 0, with "sent": false and an error under upstream. Without --agent, the command prints feedback recorded locally (N chars) and an upstream POST: line instead.
4. List local entries
./acme-orders feedback list --json --limit 3
[
{
"text": "orders list --status accepts one value only; agents tried a comma-separated list",
"cli": "acme-orders",
"version": "1.0.0"
}
]
The local file is shared by every build of acme-orders on the machine, so it can also list entries from earlier builds.
5. Find your entry in the maintainer inbox
Read the inbox as JSON and look for your marker, retrying for up to 30 seconds:
for i in 1 2 3 4 5 6; do
n=$(okibi clis feedback acme-orders --json | python3 -c "import json,sys;print(sum('marker-$M' in e['text'] for e in json.load(sys.stdin)['entries']))")
[ "$n" -ge 1 ] && break; sleep 5
done; echo "entries with the marker: $n"
entries with the marker: 1
The entry arrived on the first read. It has the same text, "cliName": "acme-orders", "version": "1.0.0" and a null agentId. The inbox command returns the latest entries only (100 by default, never more than 200), so the count above is the number of entries returned, not the size of the whole inbox.
6. Check usage
okibi clis insights acme-orders --json
{ "section": "insights", "commandAnalyticsEnabled": false, "uniqueUsers": 0, "successfulCommands": 0, "installerFetches": 2 }
The real output is larger than this sample. It also has commandAnalytics, commandAnalyticsBuild and commandEvents. Read the three counters like this:
uniqueUsersandsuccessfulCommandscount verified Okibi Identity users and invocations. They are separate from anonymous command analytics, and Identity is not required to turn those reports on.installerFetchescounts fetches of the installer script. It is recorded when the script is served, before anything knows whether the installation completes, and recording is best effort. Failures, retries and concurrent fetches change it. It is not a count of unique users or of successful installs. It was 2 after two signed installs here, and 1 on a CLI with one install, which are observations and not a guarantee that it rises by exactly one per install.
For the top-line summary over 1, 7 or 30 days:
okibi clis insights overview acme-orders --json
The relevant fields are analyticsEnabled (false), newFeedback.count (counts new inbox entries in the period) and analytics.calls (0), plus a per-day trend array.
Verify the result
M=$(date +%s)
./acme-orders feedback "verify check [marker-$M]" --agent | python3 -c "import json,sys;d=json.load(sys.stdin);print('recorded',d['recorded'],'sent',d['upstream']['sent'])"
for i in 1 2 3 4 5 6; do
n=$(okibi clis feedback acme-orders --json | python3 -c "import json,sys;print(sum('marker-$M' in e['text'] for e in json.load(sys.stdin)['entries']))")
[ "$n" -ge 1 ] && break; sleep 5
done; echo "entries with the marker: $n"
Expected: recorded True sent True, then entries with the marker: 1. The marker is new on every run, so an entry from an earlier run cannot satisfy the check, and the expected number is the same every time.
How it works
The feedback command writes to a local JSONL file first and then posts to the publisher's feedback endpoint. The okibi clis feedback command reads that endpoint's inbox for the CLI. insights reads the installer-script fetch count, the Identity counters and, when command analytics are enabled for the CLI, anonymous command reports.
Common issues
Command usage stayed at zero
After two installs, the command analytics were empty, even though commands were run from the built and installed CLIs, because command analytics were off for this CLI ("commandAnalyticsEnabled": false). With analytics off, insights commands, groups and events returned an empty page with "isDone": true, insights common returned "patterns": [] with 0 examined groups, and insights workflows returned {"status": "missing"}.
To get data, turn command analytics on and wait for the rebuild:
okibi clis analytics enable acme-orders --json okibi clis analytics status acme-orders --json
{ "enabled": true, "ok": true }
{ "build": { "status": "pending" }, "enabled": true } # about a minute later:
{ "build": { "status": "ready" }, "enabled": true }
On 0.2.11 the build took about 50 seconds: completedAt was 52 seconds after requestedAt, and status read ready after about 48 seconds. Poll analytics status until it flips. Enabling analytics built a new release (1.0.3, kind configuration-update), and the official release stayed 1.0.2 until a newer release was promoted. After installing an official release and running six commands from it (one 404, one invalid status), okibi clis insights overview acme-orders --json showed analytics.calls 7 (the six plus doctor), successes 5, failures 2, errorRate 0.29 and p50Ms 100. okibi clis insights commands acme-orders --json listed each command with calls, failures, p50 and its error categories (http with a 404 count, and unknown for the invalid status). uniqueUsers and successfulCommands still read 0. They are the Identity counters described in step 6, and enabling command analytics does not change them.
--limit is bounded
A limit of 0 or below is dropped by the client, so the server's default of 100 applies, and the server clamps any larger limit to 200. --limit 0, --limit -1 and --limit 500 all returned the same list as the default (7 entries in the test inbox), and --limit 1 returned 1. The command never returns more than 200 entries, so a count of returned entries is not a count of the whole inbox.
Entries are shared with the publisher
Feedback is sent upstream unless you pass --local-only or set ACME_ORDERS_FEEDBACK_LOCAL_ONLY=true. Do not put secrets in the text.
The MCP server also has a feedback tool
The MCP surface has a send_feedback tool that does the same job. The local-only controls (--local-only, ACME_ORDERS_FEEDBACK_LOCAL_ONLY) apply to the CLI command only: the MCP tool posts upstream even when the variable is set. Its message argument is required, and tool_name, expected and actual are optional. Called with a message, it returns {"sent": true, "truncated": false}, and the entry reaches the same hosted inbox within a few seconds. With no message it returns feedback text is empty as an error.
verification: status: verified tested_at: "2026-11-02" product_version: "okibi 0.2.11; acme-orders 1.0.0" command: "M=$(date +%s); ./acme-orders feedback \"verify check [marker-$M]\" --agent" expected_result: "recorded true and upstream.sent true, then the entry with marker-$M is found in okibi clis feedback acme-orders --json"
