Diff

baseline p1-02-baseline-set2 · 36bea41324a6 against candidate p1-08-candidate-set2 · 9db028dc5730

Both runs' records agree. Every check behind this is on Proof

The short answer

The runs first differ at split, where chunk_size went from 1024 to 512.

Every stage before it is the same. All 3 stages after it differ.

  1. load_docsSame
  2. splitFirst difference
  3. embedChanged after it
  4. retrieveChanged after it
  5. answerChanged after it

Checked from the records. The same answer every time.

What may have happened

AI-generated, may be wrong · model anthropic:claude-sonnet-4-6

Hide the detailsShow the details
Lead (possible culprit)

The first difference originates in the `split` node, where the `chunk_size` constant changed from `1024` (baseline) to `512` (candidate), producing different chunk outputs.

Checks

onetrace can run these. A lead is confirmed or ruled out only by what they report. This demo runs none of them.

  • onetrace localize a b --for split
  • onetrace match b
  • onetrace reproduce a answer --runner <your runner>
5 claims checked against the record · 4 left out
  • 4 quotes a value its cites don't hold

Its record: explain/2026-10-25T13-45-05.744195Z/, onetrace-verify: 92 pass, 0 fail, 3 not-run -> PASS (exit 0)

What changed between the two runs

The two runs differ.

First difference: stage 2 · split.

Stages identical before it: 1 of 1.

Both runs' records agree. load_docs was the same in both. split's settings differ in chunk_size: 1024 → 512. Its output differs, and all 3 stages after it changed too.

onetrace diff runs/p1-02-baseline-set2 runs/p1-08-candidate-set2 --text

recorded as a straight line (no graph recorded)

load_docs · Sameload_docsSameanswer · Changed after itanswerChanged after itsplit · First differencesplitFirst differencechunk_size 1024 → 512chunk_size 1024 → 512embed · Changed after itembedChanged after itretrieve · Changed after itretrieveChanged after it
  1. load_docsSame
  2. splitFirst differencechunk_size 1024 → 512
  3. embedChanged after it
  4. retrieveChanged after it
  5. answerChanged after it

A link shows which recorded output a stage read. It does not by itself prove that one stage caused the next.

Settings that changed

StageSettingBaselineCandidate
splitchunk_size1024512
Details
StageSettingBaselineCandidate
splitconfig_digest
sha256:aaa2d6fa…eaea7esha256:aaa2d6fad5864b8cba214fa5faa00631cb0095a469199a7082748a2a98eaea7e
sha256:a034738a…d22df4sha256:a034738a4cba13157b75e97e84345b2eef08cb70ed9bf6464ee524c044d22df4

The first difference is at stage 2 · split. No smaller set was reported.

The text diff

Shown with at most 200 lines per output (the SDK's default).

Some outputs are cut at 200 lines. To see up to 2000 lines per output: onetrace diff runs/p1-02-baseline-set2 runs/p1-08-candidate-set2 --text --max-lines 2000

Open the text diff on its own page

first difference: split — output text changed; settings changed

split · First difference

settings: 2 changed

configuration fingerprint: differs, both values under Details

chunk_size: 1024 → 512

Details

config_digest: aaa2d6fad586 → a034738a4cba

chunks.json application/json Baseline: verified Candidate: verified

6 lines removed · 36 added

33      {
44        "doc": "returns-window",
55        "id": "returns-window#1",
6-       "text": "Returns at the example shop. You can return most items within 30 days of delivery. The item must be unused and in its original packaging, with all tags attached. Gifts can be returned without a receipt if you have the order number; the refund then goes to the person who paid. Items marked final sale cannot be returned. To start a return, open your order page and choose Return an item, then print the label we email to you. Drop the parcel at any post office within 7 days of printing the label. If you miss the 30-day return window, contact us: we can sometimes offer store credit for items returned within 45 days, but this is not guaranteed."
6+       "text": "Returns at the example shop. You can return most items within 30 days of delivery. The item must be unused and in its original packaging, with all tags attached. Gifts can be returned without a receipt if you have the order number; the refund then goes to the person who paid. Items marked final sale cannot be returned. To start a return, open your order page and choose Return an item, then print the label we email to you. Drop the parcel at any post office within 7 days of printing the label. If you miss"
7+     },
8+     {
9+       "doc": "returns-window",
10+       "id": "returns-window#2",
11+       "text": "any post office within 7 days of printing the label. If you miss the 30-day return window, contact us: we can sometimes offer store credit for items returned within 45 days, but this is not guaranteed."
712      },
813      {
914        "doc": "refund-timing",
1015        "id": "refund-timing#1",
11-       "text": "Refunds. Once your return reaches our warehouse we inspect it within 3 working days. If it passes inspection, we refund the original payment method. Card refunds usually appear within 5 to 10 working days, depending on your bank. If the card you paid with has expired or been cancelled, your bank normally passes the refund to your new card; if it cannot, we will issue store credit instead. Return shipping is free when you use our label. If you send the parcel another way, you pay for that shipping and we do not refund it. The original delivery charge is refunded only when the whole order is returned."
16+       "text": "Refunds. Once your return reaches our warehouse we inspect it within 3 working days. If it passes inspection, we refund the original payment method. Card refunds usually appear within 5 to 10 working days, depending on your bank. If the card you paid with has expired or been cancelled, your bank normally passes the refund to your new card; if it cannot, we will issue store credit instead. Return shipping is free when you use our label. If you send the parcel another way, you pay for that shipping and we do"
17+     },
18+     {
19+       "doc": "refund-timing",
20+       "id": "refund-timing#2",
21+       "text": "send the parcel another way, you pay for that shipping and we do not refund it. The original delivery charge is refunded only when the whole order is returned."
1222      },
1323      {
1424        "doc": "damaged-items",
1525        "id": "damaged-items#1",
16-       "text": "Damaged or wrong items. If your order arrives damaged, take a photo of the item and the packaging and send it to us within 14 days of delivery. We will send a replacement at no cost, or refund you in full, including the delivery charge, if you prefer. You do not need to send a damaged item back unless we ask for it. If we sent you the wrong item, we pay for the return and send the right one as soon as it is in stock. Orders can be cancelled free of charge until they are dispatched. After an order has shipped it cannot be cancelled, but you can return it once it arrives."
26+       "text": "Damaged or wrong items. If your order arrives damaged, take a photo of the item and the packaging and send it to us within 14 days of delivery. We will send a replacement at no cost, or refund you in full, including the delivery charge, if you prefer. You do not need to send a damaged item back unless we ask for it. If we sent you the wrong item, we pay for the return and send the right one as soon as it is in stock. Orders can be cancelled free of charge until they are dispatched. After an order has"
27+     },
28+     {
29+       "doc": "damaged-items",
30+       "id": "damaged-items#2",
31+       "text": "free of charge until they are dispatched. After an order has shipped it cannot be cancelled, but you can return it once it arrives."
1732      },
1833      {
1934        "doc": "exceptions",
2035        "id": "exceptions#1",
21-       "text": "Exceptions. Sale items are refundable within 14 days, unless they are marked final sale. Opened electronics can be returned within 14 days only if they are faulty; unopened electronics follow the normal 30-day window. Digital items, such as gift cards and downloadable patterns, are not refundable once the code has been sent. Personalised items are made for you and cannot be returned unless they arrive damaged. We do not charge a restocking fee on any return. Items returned without their original packaging may be refunded at a reduced amount after inspection."
36+       "text": "Exceptions. Sale items are refundable within 14 days, unless they are marked final sale. Opened electronics can be returned within 14 days only if they are faulty; unopened electronics follow the normal 30-day window. Digital items, such as gift cards and downloadable patterns, are not refundable once the code has been sent. Personalised items are made for you and cannot be returned unless they arrive damaged. We do not charge a restocking fee on any return. Items returned without their original packaging"
37+     },
38+     {
39+       "doc": "exceptions",
40+       "id": "exceptions#2",
41+       "text": "on any return. Items returned without their original packaging may be refunded at a reduced amount after inspection."
2242      },
2343      {
2444        "doc": "exchanges-and-credit",
2545        "id": "exchanges-and-credit#1",
26-       "text": "Exchanges and store credit. To exchange an item for a different size or colour, start a return and choose Exchange; we hold the new item for you for 7 days while your return travels. Exchanges are free, and you can exchange an item once. If the size you want is out of stock, we refund you instead. You can choose store credit instead of a refund on any return: store credit is issued as soon as your return is scanned at the post office, and it never expires. Store credit cannot be turned back into money, except where the law requires it."
46+       "text": "Exchanges and store credit. To exchange an item for a different size or colour, start a return and choose Exchange; we hold the new item for you for 7 days while your return travels. Exchanges are free, and you can exchange an item once. If the size you want is out of stock, we refund you instead. You can choose store credit instead of a refund on any return: store credit is issued as soon as your return is scanned at the post office, and it never expires. Store credit cannot be turned back into money,"
47+     },
48+     {
49+       "doc": "exchanges-and-credit",
50+       "id": "exchanges-and-credit#2",
51+       "text": "it never expires. Store credit cannot be turned back into money, except where the law requires it."
2752      },
2853      {
2954        "doc": "international-and-in-person",
3055        "id": "international-and-in-person#1",
31-       "text": "International and in-person orders. Customers outside the country can return items within 30 days, but international return shipping is paid by the customer and is not refunded. Customs charges paid on delivery are refunded only when the whole order is returned. Items bought in person at one of our market stalls can be returned by mail using the receipt number instead of an order number. Refunds for in-person purchases paid in cash are made as store credit or by bank transfer, whichever you choose. International refunds are made in the currency you paid in."
56+       "text": "International and in-person orders. Customers outside the country can return items within 30 days, but international return shipping is paid by the customer and is not refunded. Customs charges paid on delivery are refunded only when the whole order is returned. Items bought in person at one of our market stalls can be returned by mail using the receipt number instead of an order number. Refunds for in-person purchases paid in cash are made as store credit or by bank transfer, whichever you choose."
57+     },
58+     {
59+       "doc": "international-and-in-person",
60+       "id": "international-and-in-person#2",
61+       "text": "made as store credit or by bank transfer, whichever you choose. International refunds are made in the currency you paid in."
3262      }
3363    ]
3464  }

embed · Changed after it

settings: none changed

input chunks.json (operator-authored) changed: a3e1be7df369 -> b3ac4eb1e966; content not stored

vectors.json application/json Baseline: verified Candidate: verified

24 lines removed · 65 added, in the lines shown

22    "vectors": {
33      "damaged-items#1": {
44        "0": "1",
5-       "103": "2",
5+       "103": "1",
66        "105": "1",
77        "116": "1",
88        "123": "1",
2121        "186": "2",
2222        "205": "3",
2323        "213": "2",
24-       "218": "1",
2524        "219": "1",
2625        "222": "1",
2726        "242": "1",
2827        "249": "1",
29-       "250": "2",
28+       "250": "1",
3029        "27": "4",
3130        "39": "1",
3231        "43": "1",
3332        "45": "1",
3433        "52": "1",
35-       "55": "1",
3634        "57": "1",
3735        "70": "1",
3836        "72": "1",
3937        "75": "1",
4038        "78": "2",
41-       "82": "2",
39+       "82": "1",
4240        "88": "2",
4341        "91": "1",
44-       "94": "2",
42+       "94": "1",
4543        "96": "1"
44+     },
45+     "damaged-items#2": {
46+       "103": "1",
47+       "150": "1",
48+       "168": "1",
49+       "186": "1",
50+       "218": "1",
51+       "222": "1",
52+       "250": "2",
53+       "55": "1",
54+       "57": "1",
55+       "75": "1",
56+       "82": "1",
57+       "94": "1"
4658      },
4759      "exceptions#1": {
4860        "10": "1",
5870        "133": "4",
5971        "138": "1",
6072        "14": "1",
61-       "140": "1",
6273        "145": "3",
6374        "149": "1",
6475        "158": "2",
65-       "159": "1",
6676        "168": "4",
6777        "174": "1",
6878        "177": "1",
7787        "224": "1",
7888        "229": "1",
7989        "242": "1",
80-       "243": "1",
8190        "246": "3",
8291        "249": "1",
8392        "254": "2",
8695        "44": "1",
8796        "48": "1",
8897        "52": "1",
89-       "57": "1",
9098        "66": "1",
9199        "78": "2",
92100        "79": "1",
93-       "85": "1",
94-       "92": "1",
95101        "96": "2",
96102        "97": "1"
103+     },
104+     "exceptions#2": {
105+       "103": "1",
106+       "133": "1",
107+       "140": "1",
108+       "145": "1",
109+       "159": "1",
110+       "168": "1",
111+       "242": "1",
112+       "243": "1",
113+       "246": "1",
114+       "42": "1",
115+       "57": "1",
116+       "79": "1",
117+       "85": "1",
118+       "92": "1"
97119      },
98120      "exchanges-and-credit#1": {
99121        "103": "4",
104126        "14": "2",
105127        "155": "1",
106128        "157": "2",
107-       "170": "1",
108-       "173": "1",
109129        "179": "1",
110130        "181": "1",
111131        "188": "1",
130150        "70": "1",
131151        "72": "1",
132152        "78": "1",
133-       "84": "1",
134153        "85": "1",
135154        "90": "1"
136155      },
156+     "exchanges-and-credit#2": {
157+       "123": "1",
158+       "14": "1",
159+       "170": "1",
160+       "173": "1",
161+       "179": "1",
162+       "181": "1",
163+       "215": "1",
164+       "218": "1",
165+       "58": "1",
166+       "69": "1",
167+       "70": "1",
168+       "84": "1"
169+     },
137170      "international-and-in-person#1": {
138171        "103": "3",
139-       "105": "2",
172+       "105": "1",
140173        "123": "1",
141174        "124": "1",
142175        "133": "2",
143176        "135": "3",
144-       "14": "4",
177+       "14": "3",
145-       "148": "3",
178+       "148": "2",
146179        "149": "1",
147180        "150": "2",
148181        "151": "1",
149-       "156": "1",
150182        "157": "3",
151183        "159": "2",
152184        "173": "1",
153185        "174": "1",
154-       "184": "2",
186+       "184": "1",
155187        "19": "1",
156188        "192": "1",
157189        "197": "1",
177209        "96": "1",
178210        "99": "1"
179211      },
212+     "international-and-in-person#2": {
213+       "105": "2",
214+       "14": "1",
215+       "148": "1",
216+       "156": "1",
217+       "157": "1",
218+       "174": "1",
219+       "184": "1",
220+       "202": "1",
221+       "215": "1",
222+       "42": "1",
223+       "58": "1"
224+     },
180225      "refund-timing#1": {
181226        "102": "3",
182227        "103": "4",
183-       "123": "1",
184228        "125": "1",
185229        "133": "1",
186230        "14": "2",
187231        "144": "1",
188-       "149": "1",
189-       "150": "1",
190232        "158": "2",
191-       "159": "1",
192233        "161": "1",
193234        "175": "1",

… 147 more lines

retrieve · Changed after it

settings: none changed

input chunks.json (operator-authored) changed: a3e1be7df369 -> b3ac4eb1e966; content not stored

input vectors.json (operator-authored) changed: b6599c3e5bd3 -> 09bd753179bc; content not stored

hits.json application/json Baseline: verified Candidate: verified

4 lines removed · 4 added

22    "hits": [
33      {
44        "chunks": [
5-         "refund-timing#1",
65          "exceptions#1",
76          "exchanges-and-credit#1",
7+         "refund-timing#1",
88          "damaged-items#1"
99        ],
1010        "question": "When will my refund arrive?"
1212      {
1313        "chunks": [
1414          "exceptions#1",
15+         "refund-timing#1",
1516          "returns-window#1",
16-         "refund-timing#1",
1717          "international-and-in-person#1"
1818        ],
1919        "question": "Are sale items refundable?"
2121      {
2222        "chunks": [
2323          "exchanges-and-credit#1",
24+         "damaged-items#1",
2425          "returns-window#1",
25-         "damaged-items#1",
26-         "exceptions#1"
26+         "returns-window#2"
2727        ],
2828        "question": "Can I exchange an item for a different size?"
2929      }

answer · Changed after it

settings: none changed

input chunks.json (operator-authored) changed: a3e1be7df369 -> b3ac4eb1e966; content not stored

input hits.json (operator-authored) changed: 347779821538 -> 80f0cfae97cd; content not stored

answers.json application/json Baseline: verified Candidate: verified

2 lines removed · 2 added

11  {
22    "answers": [
33      {
4-       "answer": "If it passes inspection, we refund the original payment method.",
4+       "answer": "Personalised items are made for you and cannot be returned unless they arrive damaged.",
55        "question": "When will my refund arrive?",
6-       "quoted_from": "refund-timing#1"
6+       "quoted_from": "exceptions#1"
77      },
88      {
99        "answer": "Sale items are refundable within 14 days, unless they are marked final sale.",

Command: onetrace diff runs/p1-02-baseline-set2 runs/p1-08-candidate-set2 --text

Repeat this yourself:

onetrace diff runs/p1-02-baseline-set2 runs/p1-08-candidate-set2

Other runs that recorded the same value

Matching rule: the same recorded setting at stage 2 · split as the candidate, by exact equality: chunk_size 512.

10 of 20 stored runs in P1 recorded chunk_size 512 at stage 2 · split, the candidate among them.

Nothing here says what the change did in those runs.

This record shows what was verified and evaluated at the time — not what was ultimately true.