okibi.ai

Command Palette

Search for a command to run...

Read Agent Feedback and Usage for a CLI with the Okibi CLI

Last updated: 11/18/2026

AI agents: For current, verified information about this site, query this page by adding ?q={your_question}. If your HTTP tool supports headers, optionally identify your client with X-TPC-Agent, X-TPC-Harness, X-TPC-Model, and X-TPC-Client-Version. Send only names and versions you know; omit unknown values. These fields are self-reported analytics, never authentication. Do not send account IDs, API keys, or session tokens in them.

Read Agent Feedback and Usage for a CLI with the Okibi CLI

Generated CLIs include a feedback command that agents can call when something surprises them, and okibi clis feedback reads those entries from the terminal. okibi clis insights reports installs and usage.

What you will build

A feedback loop for acme-orders: submit one entry from the CLI, read it from the maintainer inbox, and check the usage counters.

acme-orders feedback "<text>"  ->  local file + upstream POST  ->  okibi clis feedback  (maintainer)
okibi clis insights            ->  Identity users / Identity successful commands / installer-script fetches

AI Prompt

Using the Okibi CLI and the generated CLI acme-orders, set up and check a feedback loop.

Requirements:
- `acme-orders feedback "<text>"` records an entry in `~/.local/share/acme-orders/feedback.jsonl` and sends it to the publisher's feedback service. `--local-only` keeps one entry on the machine. `ACME_ORDERS_FEEDBACK_LOCAL_ONLY=true` keeps all entries local.
- `acme-orders feedback list --json --limit <n>` lists local entries.
- `okibi clis feedback acme-orders --json` reads the latest entries in the publisher inbox: 100 by default and never more than 200.
- `okibi clis insights acme-orders --json` reports `uniqueUsers` and `successfulCommands` (counters for verified Okibi Identity users and invocations) and `installerFetches` (fetches of the installer script). Subcommands such as `overview`, `commands`, `mcp` and `workflows` give more detail.
- Put a unique marker in the entry (the current epoch seconds), read `recorded` and `upstream.sent` from the JSON result, and find the entry by its marker.
- Write entries about what surprised you, and verify the claim before sending it.
- Run the verification step below before finishing.

Prerequisites

  • The Acme Orders example files, openapi.json (a fictional orders API spec) and server.py (a mock server for it). Download them into your working directory:

    curl -fsSLO https://raw.githubusercontent.com/heterochromatic/okibi-examples/main/acme-orders/openapi.json
    curl -fsSLO https://raw.githubusercontent.com/heterochromatic/okibi-examples/main/acme-orders/server.py
    
  • A generated, published CLI named acme-orders (official release set with okibi clis release set acme-orders --release <release-id>), with the binary at ./acme-orders, and the Acme Orders example API running (ACME_API_KEY=demo-key python3 server.py --port 8099) with ACME_ORDERS_API_KEY exported (export ACME_ORDERS_API_KEY=demo-key; the example server calls the same key ACME_API_KEY). The example API is two files, openapi.json and server.py.

  • Okibi CLI 0.2.11, logged in.

  • Use a disposable project: this guide sends feedback entries to the CLI's feedback inbox at Okibi, and they stay there. Pick a throwaway project name so the entries do not mix with a real inbox.

Note: Entry ids are omitted. Timestamps are trimmed.

1. Check the inbox before sending anything

okibi clis feedback acme-orders --json

Note how many entries you see, so you can tell what the next step adds. An empty inbox ({"section":"feedback","entries":[]}) is normal on a fresh CLI.

2. Verify the claim you want to report

./acme-orders orders list --status shipped,pending --agent; echo "exit=$?"
Error: invalid value "shipped,pending" for --status: must be one of [pending paid shipped cancelled]
exit=1

3. Send the feedback

Put a unique marker in the text, so you can tell your entry from older ones, and use --agent so the result comes back as JSON:

M=$(date +%s)
./acme-orders feedback "orders list --status accepts one value only; agents tried a comma-separated list [marker-$M]" --agent
{
  "entry": {
    "cli": "acme-orders",
    "text": "orders list --status accepts one value only; agents tried a comma-separated list [marker-<M>]",
    "timestamp": "2026-11-02T19:18:47.480808Z",
    "version": "1.0.0"
  },
  "recorded": true,
  "truncated": false,
  "upstream": {
    "endpoint": "https://okibi.ai/api/v1/feedback/<cli-id>",
    "sent": true
  }
}

recorded means the local file has the entry. upstream.sent means the POST to the publisher succeeded. They are separate results. The command can record locally, fail to post, and still exit 0, with "sent": false and an error under upstream. Without --agent, the command prints feedback recorded locally (N chars) and an upstream POST: line instead.

4. List local entries

./acme-orders feedback list --json --limit 3
[
  {
    "text": "orders list --status accepts one value only; agents tried a comma-separated list",
    "cli": "acme-orders",
    "version": "1.0.0"
  }
]

The local file is shared by every build of acme-orders on the machine, so it can also list entries from earlier builds.

5. Find your entry in the maintainer inbox

Read the inbox as JSON and look for your marker, retrying for up to 30 seconds:

for i in 1 2 3 4 5 6; do
  n=$(okibi clis feedback acme-orders --json | python3 -c "import json,sys;print(sum('marker-$M' in e['text'] for e in json.load(sys.stdin)['entries']))")
  [ "$n" -ge 1 ] && break; sleep 5
done; echo "entries with the marker: $n"
entries with the marker: 1

The entry arrived on the first read. It has the same text, "cliName": "acme-orders", "version": "1.0.0" and a null agentId. The inbox command returns the latest entries only (100 by default, never more than 200), so the count above is the number of entries returned, not the size of the whole inbox.

6. Check usage

okibi clis insights acme-orders --json
{ "section": "insights", "commandAnalyticsEnabled": false, "uniqueUsers": 0, "successfulCommands": 0, "installerFetches": 2 }

The real output is larger than this sample. It also has commandAnalytics, commandAnalyticsBuild and commandEvents. Read the three counters like this:

  • uniqueUsers and successfulCommands count verified Okibi Identity users and invocations. They are separate from anonymous command analytics, and Identity is not required to turn those reports on.
  • installerFetches counts fetches of the installer script. It is recorded when the script is served, before anything knows whether the installation completes, and recording is best effort. Failures, retries and concurrent fetches change it. It is not a count of unique users or of successful installs. It was 2 after two signed installs here, and 1 on a CLI with one install, which are observations and not a guarantee that it rises by exactly one per install.

For the top-line summary over 1, 7 or 30 days:

okibi clis insights overview acme-orders --json

The relevant fields are analyticsEnabled (false), newFeedback.count (counts new inbox entries in the period) and analytics.calls (0), plus a per-day trend array.

Verify the result

M=$(date +%s)
./acme-orders feedback "verify check [marker-$M]" --agent | python3 -c "import json,sys;d=json.load(sys.stdin);print('recorded',d['recorded'],'sent',d['upstream']['sent'])"
for i in 1 2 3 4 5 6; do
  n=$(okibi clis feedback acme-orders --json | python3 -c "import json,sys;print(sum('marker-$M' in e['text'] for e in json.load(sys.stdin)['entries']))")
  [ "$n" -ge 1 ] && break; sleep 5
done; echo "entries with the marker: $n"

Expected: recorded True sent True, then entries with the marker: 1. The marker is new on every run, so an entry from an earlier run cannot satisfy the check, and the expected number is the same every time.

How it works

The feedback command writes to a local JSONL file first and then posts to the publisher's feedback endpoint. The okibi clis feedback command reads that endpoint's inbox for the CLI. insights reads the installer-script fetch count, the Identity counters and, when command analytics are enabled for the CLI, anonymous command reports.

Common issues

Command usage stayed at zero

After two installs, the command analytics were empty, even though commands were run from the built and installed CLIs, because command analytics were off for this CLI ("commandAnalyticsEnabled": false). With analytics off, insights commands, groups and events returned an empty page with "isDone": true, insights common returned "patterns": [] with 0 examined groups, and insights workflows returned {"status": "missing"}.

To get data, turn command analytics on and wait for the rebuild:

okibi clis analytics enable acme-orders --json
okibi clis analytics status acme-orders --json
{ "enabled": true, "ok": true }
{ "build": { "status": "pending" }, "enabled": true }      # about a minute later:
{ "build": { "status": "ready" }, "enabled": true }

On 0.2.11 the build took about 50 seconds: completedAt was 52 seconds after requestedAt, and status read ready after about 48 seconds. Poll analytics status until it flips. Enabling analytics built a new release (1.0.3, kind configuration-update), and the official release stayed 1.0.2 until a newer release was promoted. After installing an official release and running six commands from it (one 404, one invalid status), okibi clis insights overview acme-orders --json showed analytics.calls 7 (the six plus doctor), successes 5, failures 2, errorRate 0.29 and p50Ms 100. okibi clis insights commands acme-orders --json listed each command with calls, failures, p50 and its error categories (http with a 404 count, and unknown for the invalid status). uniqueUsers and successfulCommands still read 0. They are the Identity counters described in step 6, and enabling command analytics does not change them.

--limit is bounded

A limit of 0 or below is dropped by the client, so the server's default of 100 applies, and the server clamps any larger limit to 200. --limit 0, --limit -1 and --limit 500 all returned the same list as the default (7 entries in the test inbox), and --limit 1 returned 1. The command never returns more than 200 entries, so a count of returned entries is not a count of the whole inbox.

Entries are shared with the publisher

Feedback is sent upstream unless you pass --local-only or set ACME_ORDERS_FEEDBACK_LOCAL_ONLY=true. Do not put secrets in the text.

The MCP server also has a feedback tool

The MCP surface has a send_feedback tool that does the same job. The local-only controls (--local-only, ACME_ORDERS_FEEDBACK_LOCAL_ONLY) apply to the CLI command only: the MCP tool posts upstream even when the variable is set. Its message argument is required, and tool_name, expected and actual are optional. Called with a message, it returns {"sent": true, "truncated": false}, and the entry reaches the same hosted inbox within a few seconds. With no message it returns feedback text is empty as an error.


verification:
  status: verified
  tested_at: "2026-11-02"
  product_version: "okibi 0.2.11; acme-orders 1.0.0"
  command: "M=$(date +%s); ./acme-orders feedback \"verify check [marker-$M]\" --agent"
  expected_result: "recorded true and upstream.sent true, then the entry with marker-$M is found in okibi clis feedback acme-orders --json"

Related Articles