Skip to content

Instantly share code, notes, and snippets.

@a-robinson

a-robinson/gist:14d7740fd58c221cd2bb089d5b18eed8

Created June 4, 2018 20:38
Show Gist options
  • Save a-robinson/14d7740fd58c221cd2bb089d5b18eed8 to your computer and use it in GitHub Desktop.
Save a-robinson/14d7740fd58c221cd2bb089d5b18eed8 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.txt
This file has been truncated, but you can view the full file.
root@:26257/defaultdb> show trace for show testing_ranges from table foo;
+----------------------------------+----------------------+--------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------+---------------------------------+-------------------------------------------------------+------+
| timestamp | age | message | tag | loc | operation | span |
+----------------------------------+----------------------+--------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------+---------------------------------+-------------------------------------------------------+------+
| 2018-06-04 19:47:09.788057+00:00 | 0s | === SPAN START: sql txn === | | | sql txn | 0 |
| 2018-06-04 19:47:09.79127+00:00 | 3ms212µs861ns | === SPAN START: session recording === | | | session recording | 507 |
| 2018-06-04 19:47:09.791273+00:00 | 3ms216µs116ns | === SPAN START: starting plan === | | | starting plan | 1 |
| 2018-06-04 19:47:09.791288+00:00 | 3ms231µs310ns | === SPAN START: dist sender === | | | dist sender | 2 |
| 2018-06-04 19:47:09.791309+00:00 | 3ms252µs718ns | querying next range at /Meta2/Table/52/1/NULL | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 2 |
| 2018-06-04 19:47:09.791336+00:00 | 3ms279µs4ns | === SPAN START: range lookup === | | | range lookup | 3 |
| 2018-06-04 19:47:09.791358+00:00 | 3ms301µs331ns | === SPAN START: dist sender === | | | dist sender | 4 |
| 2018-06-04 19:47:09.791387+00:00 | 3ms330µs548ns | querying next range at /Meta1/Table/52/1/NULL/NULL | [client=[::1]:57501,user=root,txn=9352587e,range-lookup=/Meta2/Table/52/1/NULL,n1] | | dist sender | 4 |
| 2018-06-04 19:47:09.791403+00:00 | 3ms346µs358ns | === SPAN START: range lookup === | | | range lookup | 5 |
| 2018-06-04 19:47:09.791434+00:00 | 3ms376µs850ns | looked up range descriptor | [client=[::1]:57501,user=root,txn=9352587e,range-lookup=/Meta2/Table/52/1/NULL,n1] | | dist sender | 4 |
| 2018-06-04 19:47:09.791457+00:00 | 3ms399µs881ns | r1: sending batch 1 Scan to (n2,s2):3 | [client=[::1]:57501,user=root,txn=9352587e,range-lookup=/Meta2/Table/52/1/NULL,n1] | | dist sender | 4 |
| 2018-06-04 19:47:09.791489+00:00 | 3ms432µs557ns | sending request to alex-laptop.local:26259 | [client=[::1]:57501,user=root,txn=9352587e,range-lookup=/Meta2/Table/52/1/NULL,n1] | | dist sender | 4 |
| 2018-06-04 19:47:09.791497+00:00 | 3ms439µs882ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 6 |
| 2018-06-04 19:47:09.791798+00:00 | 3ms740µs828ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 396 |
| 2018-06-04 19:47:09.79183+00:00 | 3ms773µs661ns | 1 Scan | [n2] | | /cockroach.roachpb.Internal/Batch | 396 |
| 2018-06-04 19:47:09.791838+00:00 | 3ms781µs780ns | executing 1 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 396 |
| 2018-06-04 19:47:09.791846+00:00 | 3ms789µs659ns | read-only path | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 396 |
| 2018-06-04 19:47:09.791855+00:00 | 3ms798µs337ns | command queue | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 396 |
| 2018-06-04 19:47:09.791857+00:00 | 3ms800µs641ns | operation accepts inconsistent results | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 396 |
| 2018-06-04 19:47:09.791859+00:00 | 3ms802µs235ns | waiting for read lock | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 396 |
| 2018-06-04 19:47:09.792111+00:00 | 4ms54µs467ns | engine stats: {InternalDeleteSkippedCount:2 TimeBoundNumSSTs:0} | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 396 |
| 2018-06-04 19:47:09.792118+00:00 | 4ms61µs69ns | read completed | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 396 |
| 2018-06-04 19:47:09.792407+00:00 | 4ms350µs768ns | looked up range descriptor | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 2 |
| 2018-06-04 19:47:09.792433+00:00 | 4ms376µs34ns | r1: sending batch 1 Scan to (n2,s2):3 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 2 |
| 2018-06-04 19:47:09.792456+00:00 | 4ms399µs15ns | sending request to alex-laptop.local:26259 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 2 |
| 2018-06-04 19:47:09.792476+00:00 | 4ms419µs782ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 7 |
| 2018-06-04 19:47:09.792693+00:00 | 4ms636µs189ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 397 |
| 2018-06-04 19:47:09.792715+00:00 | 4ms658µs515ns | 1 Scan | [n2] | | /cockroach.roachpb.Internal/Batch | 397 |
| 2018-06-04 19:47:09.792722+00:00 | 4ms665µs121ns | executing 1 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 397 |
| 2018-06-04 19:47:09.792727+00:00 | 4ms670µs554ns | read-only path | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 397 |
| 2018-06-04 19:47:09.792735+00:00 | 4ms678µs226ns | command queue | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 397 |
| 2018-06-04 19:47:09.792741+00:00 | 4ms684µs107ns | waiting for read lock | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 397 |
| 2018-06-04 19:47:09.794156+00:00 | 6ms99µs698ns | engine stats: {InternalDeleteSkippedCount:102 TimeBoundNumSSTs:0} | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 397 |
| 2018-06-04 19:47:09.794178+00:00 | 6ms121µs645ns | read completed | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 397 |
| 2018-06-04 19:47:09.794569+00:00 | 6ms512µs355ns | === SPAN START: dist sender === | | | dist sender | 8 |
| 2018-06-04 19:47:09.794585+00:00 | 6ms528µs481ns | querying next range at /Meta2/Table/52/2 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 8 |
| 2018-06-04 19:47:09.794608+00:00 | 6ms551µs579ns | r1: sending batch 1 Scan to (n2,s2):3 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 8 |
| 2018-06-04 19:47:09.794626+00:00 | 6ms568µs997ns | sending request to alex-laptop.local:26259 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 8 |
| 2018-06-04 19:47:09.794641+00:00 | 6ms584µs426ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 9 |
| 2018-06-04 19:47:09.794826+00:00 | 6ms769µs775ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 398 |
| 2018-06-04 19:47:09.794848+00:00 | 6ms791µs511ns | 1 Scan | [n2] | | /cockroach.roachpb.Internal/Batch | 398 |
| 2018-06-04 19:47:09.794854+00:00 | 6ms797µs342ns | executing 1 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 398 |
| 2018-06-04 19:47:09.794859+00:00 | 6ms802µs499ns | read-only path | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 398 |
| 2018-06-04 19:47:09.794866+00:00 | 6ms809µs536ns | command queue | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 398 |
| 2018-06-04 19:47:09.794871+00:00 | 6ms814µs443ns | waiting for read lock | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 398 |
| 2018-06-04 19:47:09.79522+00:00 | 7ms162µs888ns | engine stats: {InternalDeleteSkippedCount:1 TimeBoundNumSSTs:0} | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 398 |
| 2018-06-04 19:47:09.795224+00:00 | 7ms167µs688ns | read completed | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 398 |
| 2018-06-04 19:47:09.795479+00:00 | 7ms422µs327ns | === SPAN START: consuming rows === | | | consuming rows | 10 |
| 2018-06-04 19:47:09.795506+00:00 | 7ms449µs450ns | === SPAN START: dist sender === | | | dist sender | 11 |
| 2018-06-04 19:47:09.795516+00:00 | 7ms459µs177ns | querying next range at /Table/52 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 11 |
| 2018-06-04 19:47:09.795537+00:00 | 7ms480µs309ns | r23: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 11 |
| 2018-06-04 19:47:09.795554+00:00 | 7ms497µs349ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 11 |
| 2018-06-04 19:47:09.795566+00:00 | 7ms509µs369ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 12 |
| 2018-06-04 19:47:09.795569+00:00 | 7ms512µs554ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 12 |
| 2018-06-04 19:47:09.795574+00:00 | 7ms517µs503ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 12 |
| 2018-06-04 19:47:09.795581+00:00 | 7ms524µs310ns | read-only path | [n1,s1,r23/1:/Table/52{-/1/0}] | | /cockroach.roachpb.Internal/Batch | 12 |
| 2018-06-04 19:47:09.795586+00:00 | 7ms528µs856ns | read has no clock uncertainty | [n1,s1,r23/1:/Table/52{-/1/0}] | | /cockroach.roachpb.Internal/Batch | 12 |
| 2018-06-04 19:47:09.795589+00:00 | 7ms532µs33ns | command queue | [n1,s1,r23/1:/Table/52{-/1/0}] | | /cockroach.roachpb.Internal/Batch | 12 |
| 2018-06-04 19:47:09.795594+00:00 | 7ms537µs133ns | waiting for read lock | [n1,s1,r23/1:/Table/52{-/1/0}] | | /cockroach.roachpb.Internal/Batch | 12 |
| 2018-06-04 19:47:09.795605+00:00 | 7ms548µs154ns | read completed | [n1,s1,r23/1:/Table/52{-/1/0}] | | /cockroach.roachpb.Internal/Batch | 12 |
| 2018-06-04 19:47:09.795642+00:00 | 7ms585µs348ns | === SPAN START: dist sender === | | | dist sender | 13 |
| 2018-06-04 19:47:09.79565+00:00 | 7ms593µs519ns | querying next range at /Table/52/1/0 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 13 |
| 2018-06-04 19:47:09.795666+00:00 | 7ms609µs506ns | r24: sending batch 1 LeaseInfo to (n2,s2):3 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 13 |
| 2018-06-04 19:47:09.795685+00:00 | 7ms628µs259ns | sending request to alex-laptop.local:26259 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 13 |
| 2018-06-04 19:47:09.795699+00:00 | 7ms642µs28ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 14 |
| 2018-06-04 19:47:09.79591+00:00 | 7ms853µs244ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 399 |
| 2018-06-04 19:47:09.79593+00:00 | 7ms873µs757ns | 1 LeaseInfo | [n2] | | /cockroach.roachpb.Internal/Batch | 399 |
| 2018-06-04 19:47:09.795936+00:00 | 7ms878µs932ns | executing 1 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 399 |
| 2018-06-04 19:47:09.795941+00:00 | 7ms884µs458ns | read-only path | [n2,s2,r24/3:/Table/52/1/{0-100}] | | /cockroach.roachpb.Internal/Batch | 399 |
| 2018-06-04 19:47:09.795948+00:00 | 7ms891µs570ns | command queue | [n2,s2,r24/3:/Table/52/1/{0-100}] | | /cockroach.roachpb.Internal/Batch | 399 |
| 2018-06-04 19:47:09.795952+00:00 | 7ms895µs485ns | waiting for read lock | [n2,s2,r24/3:/Table/52/1/{0-100}] | | /cockroach.roachpb.Internal/Batch | 399 |
| 2018-06-04 19:47:09.795961+00:00 | 7ms904µs324ns | read completed | [n2,s2,r24/3:/Table/52/1/{0-100}] | | /cockroach.roachpb.Internal/Batch | 399 |
| 2018-06-04 19:47:09.796194+00:00 | 8ms136µs938ns | === SPAN START: dist sender === | | | dist sender | 15 |
| 2018-06-04 19:47:09.796207+00:00 | 8ms150µs543ns | querying next range at /Table/52/1/100 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 15 |
| 2018-06-04 19:47:09.796233+00:00 | 8ms175µs858ns | r25: sending batch 1 LeaseInfo to (n2,s2):3 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 15 |
| 2018-06-04 19:47:09.796255+00:00 | 8ms198µs165ns | sending request to alex-laptop.local:26259 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 15 |
| 2018-06-04 19:47:09.796274+00:00 | 8ms217µs218ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 16 |
| 2018-06-04 19:47:09.796481+00:00 | 8ms424µs126ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 400 |
| 2018-06-04 19:47:09.796501+00:00 | 8ms443µs871ns | 1 LeaseInfo | [n2] | | /cockroach.roachpb.Internal/Batch | 400 |
| 2018-06-04 19:47:09.796506+00:00 | 8ms449µs407ns | executing 1 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 400 |
| 2018-06-04 19:47:09.796512+00:00 | 8ms455µs502ns | read-only path | [n2,s2,r25/3:/Table/52/1/{100-200}] | | /cockroach.roachpb.Internal/Batch | 400 |
| 2018-06-04 19:47:09.796519+00:00 | 8ms462µs88ns | command queue | [n2,s2,r25/3:/Table/52/1/{100-200}] | | /cockroach.roachpb.Internal/Batch | 400 |
| 2018-06-04 19:47:09.796523+00:00 | 8ms466µs555ns | waiting for read lock | [n2,s2,r25/3:/Table/52/1/{100-200}] | | /cockroach.roachpb.Internal/Batch | 400 |
| 2018-06-04 19:47:09.796532+00:00 | 8ms475µs548ns | read completed | [n2,s2,r25/3:/Table/52/1/{100-200}] | | /cockroach.roachpb.Internal/Batch | 400 |
| 2018-06-04 19:47:09.796749+00:00 | 8ms692µs169ns | === SPAN START: dist sender === | | | dist sender | 17 |
| 2018-06-04 19:47:09.796759+00:00 | 8ms702µs492ns | querying next range at /Table/52/1/200 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 17 |
| 2018-06-04 19:47:09.796776+00:00 | 8ms719µs211ns | r26: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 17 |
| 2018-06-04 19:47:09.796792+00:00 | 8ms735µs577ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 17 |
| 2018-06-04 19:47:09.796802+00:00 | 8ms745µs679ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 18 |
| 2018-06-04 19:47:09.796821+00:00 | 8ms764µs270ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 18 |
| 2018-06-04 19:47:09.796825+00:00 | 8ms768µs165ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 18 |
| 2018-06-04 19:47:09.79683+00:00 | 8ms773µs585ns | read-only path | [n1,s1,r26/1:/Table/52/1/{200-300}] | | /cockroach.roachpb.Internal/Batch | 18 |
| 2018-06-04 19:47:09.79685+00:00 | 8ms793µs480ns | replica.Send got error: [NotLeaseHolderError] r26: replica | [n1,s1,r26/1:/Table/52/1/{200-300}] | | /cockroach.roachpb.Internal/Batch | 18 |
| | | (n1,s1):1 not lease holder; current lease is repl=(n2,s2):3 seq=7 | | | | |
| | | start=1528141627.325771938,0 epo=2 pro=1528141627.325774415,0 | | | | |
| 2018-06-04 19:47:09.796858+00:00 | 8ms801µs594ns | *roachpb.NotLeaseHolderError | [n1] | | /cockroach.roachpb.Internal/Batch | 18 |
| 2018-06-04 19:47:09.796872+00:00 | 8ms815µs317ns | application error: [NotLeaseHolderError] r26: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 17 |
| | | not lease holder; current lease is repl=(n2,s2):3 seq=7 | | | | |
| | | start=1528141627.325771938,0 epo=2 pro=1528141627.325774415,0 | | | | |
| 2018-06-04 19:47:09.796881+00:00 | 8ms823µs906ns | error: {(err: [NotLeaseHolderError] r26: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 17 |
| | | not lease holder; current lease is repl=(n2,s2):3 seq=7 | | | | |
| | | start=1528141627.325771938,0 epo=2 pro=1528141627.325774415,0) <nil>}; | | | | |
| | | trying next peer (n2,s2):3 | | | | |
| 2018-06-04 19:47:09.796893+00:00 | 8ms836µs96ns | sending request to alex-laptop.local:26259 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 17 |
| 2018-06-04 19:47:09.796904+00:00 | 8ms847µs660ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 19 |
| 2018-06-04 19:47:09.797058+00:00 | 9ms875ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 401 |
| 2018-06-04 19:47:09.797075+00:00 | 9ms18µs435ns | 1 LeaseInfo | [n2] | | /cockroach.roachpb.Internal/Batch | 401 |
| 2018-06-04 19:47:09.79708+00:00 | 9ms22µs825ns | executing 1 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 401 |
| 2018-06-04 19:47:09.797085+00:00 | 9ms28µs640ns | read-only path | [n2,s2,r26/3:/Table/52/1/{200-300}] | | /cockroach.roachpb.Internal/Batch | 401 |
| 2018-06-04 19:47:09.797091+00:00 | 9ms34µs213ns | command queue | [n2,s2,r26/3:/Table/52/1/{200-300}] | | /cockroach.roachpb.Internal/Batch | 401 |
| 2018-06-04 19:47:09.797094+00:00 | 9ms37µs686ns | waiting for read lock | [n2,s2,r26/3:/Table/52/1/{200-300}] | | /cockroach.roachpb.Internal/Batch | 401 |
| 2018-06-04 19:47:09.797102+00:00 | 9ms45µs14ns | read completed | [n2,s2,r26/3:/Table/52/1/{200-300}] | | /cockroach.roachpb.Internal/Batch | 401 |
| 2018-06-04 19:47:09.797286+00:00 | 9ms229µs407ns | === SPAN START: dist sender === | | | dist sender | 20 |
| 2018-06-04 19:47:09.797295+00:00 | 9ms238µs618ns | querying next range at /Table/52/1/300 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 20 |
| 2018-06-04 19:47:09.797312+00:00 | 9ms254µs855ns | r27: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 20 |
| 2018-06-04 19:47:09.797326+00:00 | 9ms269µs415ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 20 |
| 2018-06-04 19:47:09.797335+00:00 | 9ms278µs361ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 21 |
| 2018-06-04 19:47:09.797338+00:00 | 9ms281µs680ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 21 |
| 2018-06-04 19:47:09.797342+00:00 | 9ms285µs83ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 21 |
| 2018-06-04 19:47:09.797346+00:00 | 9ms289µs780ns | read-only path | [n1,s1,r27/1:/Table/52/1/{300-400}] | | /cockroach.roachpb.Internal/Batch | 21 |
| 2018-06-04 19:47:09.79735+00:00 | 9ms292µs851ns | read has no clock uncertainty | [n1,s1,r27/1:/Table/52/1/{300-400}] | | /cockroach.roachpb.Internal/Batch | 21 |
| 2018-06-04 19:47:09.797352+00:00 | 9ms295µs352ns | command queue | [n1,s1,r27/1:/Table/52/1/{300-400}] | | /cockroach.roachpb.Internal/Batch | 21 |
| 2018-06-04 19:47:09.797357+00:00 | 9ms300µs353ns | waiting for read lock | [n1,s1,r27/1:/Table/52/1/{300-400}] | | /cockroach.roachpb.Internal/Batch | 21 |
| 2018-06-04 19:47:09.797364+00:00 | 9ms307µs742ns | read completed | [n1,s1,r27/1:/Table/52/1/{300-400}] | | /cockroach.roachpb.Internal/Batch | 21 |
| 2018-06-04 19:47:09.797403+00:00 | 9ms346µs195ns | === SPAN START: dist sender === | | | dist sender | 22 |
| 2018-06-04 19:47:09.79741+00:00 | 9ms353µs80ns | querying next range at /Table/52/1/400 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 22 |
| 2018-06-04 19:47:09.797422+00:00 | 9ms365µs234ns | r28: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 22 |
| 2018-06-04 19:47:09.797434+00:00 | 9ms377µs274ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 22 |
| 2018-06-04 19:47:09.797441+00:00 | 9ms384µs160ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 23 |
| 2018-06-04 19:47:09.797443+00:00 | 9ms386µs417ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 23 |
| 2018-06-04 19:47:09.797446+00:00 | 9ms389µs80ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 23 |
| 2018-06-04 19:47:09.79745+00:00 | 9ms393µs19ns | read-only path | [n1,s1,r28/1:/Table/52/1/{400-500}] | | /cockroach.roachpb.Internal/Batch | 23 |
| 2018-06-04 19:47:09.797453+00:00 | 9ms395µs898ns | read has no clock uncertainty | [n1,s1,r28/1:/Table/52/1/{400-500}] | | /cockroach.roachpb.Internal/Batch | 23 |
| 2018-06-04 19:47:09.797454+00:00 | 9ms397µs768ns | command queue | [n1,s1,r28/1:/Table/52/1/{400-500}] | | /cockroach.roachpb.Internal/Batch | 23 |
| 2018-06-04 19:47:09.797458+00:00 | 9ms401µs372ns | waiting for read lock | [n1,s1,r28/1:/Table/52/1/{400-500}] | | /cockroach.roachpb.Internal/Batch | 23 |
| 2018-06-04 19:47:09.797463+00:00 | 9ms406µs763ns | read completed | [n1,s1,r28/1:/Table/52/1/{400-500}] | | /cockroach.roachpb.Internal/Batch | 23 |
| 2018-06-04 19:47:09.79749+00:00 | 9ms433µs10ns | === SPAN START: dist sender === | | | dist sender | 24 |
| 2018-06-04 19:47:09.797496+00:00 | 9ms439µs278ns | querying next range at /Table/52/1/500 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 24 |
| 2018-06-04 19:47:09.797508+00:00 | 9ms451µs353ns | r29: sending batch 1 LeaseInfo to (n3,s3):2 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 24 |
| 2018-06-04 19:47:09.797519+00:00 | 9ms462µs706ns | sending request to alex-laptop.local:26261 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 24 |
| 2018-06-04 19:47:09.79753+00:00 | 9ms472µs892ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 25 |
| 2018-06-04 19:47:09.797767+00:00 | 9ms710µs533ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 402 |
| 2018-06-04 19:47:09.797793+00:00 | 9ms736µs171ns | 1 LeaseInfo | [n3] | | /cockroach.roachpb.Internal/Batch | 402 |
| 2018-06-04 19:47:09.797799+00:00 | 9ms741µs977ns | executing 1 requests | [n3,s3] | | /cockroach.roachpb.Internal/Batch | 402 |
| 2018-06-04 19:47:09.797806+00:00 | 9ms749µs282ns | read-only path | [n3,s3,r29/2:/Table/52/1/{500-600}] | | /cockroach.roachpb.Internal/Batch | 402 |
| 2018-06-04 19:47:09.797813+00:00 | 9ms755µs899ns | command queue | [n3,s3,r29/2:/Table/52/1/{500-600}] | | /cockroach.roachpb.Internal/Batch | 402 |
| 2018-06-04 19:47:09.797816+00:00 | 9ms759µs691ns | waiting for read lock | [n3,s3,r29/2:/Table/52/1/{500-600}] | | /cockroach.roachpb.Internal/Batch | 402 |
| 2018-06-04 19:47:09.797824+00:00 | 9ms767µs168ns | read completed | [n3,s3,r29/2:/Table/52/1/{500-600}] | | /cockroach.roachpb.Internal/Batch | 402 |
| 2018-06-04 19:47:09.798047+00:00 | 9ms990µs327ns | === SPAN START: dist sender === | | | dist sender | 26 |
| 2018-06-04 19:47:09.798056+00:00 | 9ms998µs895ns | querying next range at /Table/52/1/600 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 26 |
| 2018-06-04 19:47:09.798071+00:00 | 10ms14µs449ns | r30: sending batch 1 LeaseInfo to (n2,s2):3 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 26 |
| 2018-06-04 19:47:09.798085+00:00 | 10ms28µs171ns | sending request to alex-laptop.local:26259 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 26 |
| 2018-06-04 19:47:09.798095+00:00 | 10ms38µs490ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 27 |
| 2018-06-04 19:47:09.79827+00:00 | 10ms213µs182ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 403 |
| 2018-06-04 19:47:09.798288+00:00 | 10ms231µs23ns | 1 LeaseInfo | [n2] | | /cockroach.roachpb.Internal/Batch | 403 |
| 2018-06-04 19:47:09.798293+00:00 | 10ms235µs855ns | executing 1 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 403 |
| 2018-06-04 19:47:09.798298+00:00 | 10ms241µs280ns | read-only path | [n2,s2,r30/3:/Table/52/1/{600-700}] | | /cockroach.roachpb.Internal/Batch | 403 |
| 2018-06-04 19:47:09.798304+00:00 | 10ms246µs929ns | command queue | [n2,s2,r30/3:/Table/52/1/{600-700}] | | /cockroach.roachpb.Internal/Batch | 403 |
| 2018-06-04 19:47:09.798307+00:00 | 10ms250µs581ns | waiting for read lock | [n2,s2,r30/3:/Table/52/1/{600-700}] | | /cockroach.roachpb.Internal/Batch | 403 |
| 2018-06-04 19:47:09.798316+00:00 | 10ms259µs777ns | read completed | [n2,s2,r30/3:/Table/52/1/{600-700}] | | /cockroach.roachpb.Internal/Batch | 403 |
| 2018-06-04 19:47:09.798485+00:00 | 10ms428µs602ns | === SPAN START: dist sender === | | | dist sender | 28 |
| 2018-06-04 19:47:09.798494+00:00 | 10ms437µs413ns | querying next range at /Table/52/1/700 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 28 |
| 2018-06-04 19:47:09.798511+00:00 | 10ms454µs215ns | r31: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 28 |
| 2018-06-04 19:47:09.798525+00:00 | 10ms468µs301ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 28 |
| 2018-06-04 19:47:09.798533+00:00 | 10ms476µs720ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 29 |
| 2018-06-04 19:47:09.798536+00:00 | 10ms479µs373ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 29 |
| 2018-06-04 19:47:09.79854+00:00 | 10ms482µs884ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 29 |
| 2018-06-04 19:47:09.798544+00:00 | 10ms487µs205ns | read-only path | [n1,s1,r31/1:/Table/52/1/{700-800}] | | /cockroach.roachpb.Internal/Batch | 29 |
| 2018-06-04 19:47:09.798547+00:00 | 10ms490µs85ns | read has no clock uncertainty | [n1,s1,r31/1:/Table/52/1/{700-800}] | | /cockroach.roachpb.Internal/Batch | 29 |
| 2018-06-04 19:47:09.798549+00:00 | 10ms492µs285ns | command queue | [n1,s1,r31/1:/Table/52/1/{700-800}] | | /cockroach.roachpb.Internal/Batch | 29 |
| 2018-06-04 19:47:09.798553+00:00 | 10ms496µs204ns | waiting for read lock | [n1,s1,r31/1:/Table/52/1/{700-800}] | | /cockroach.roachpb.Internal/Batch | 29 |
| 2018-06-04 19:47:09.798559+00:00 | 10ms502µs735ns | read completed | [n1,s1,r31/1:/Table/52/1/{700-800}] | | /cockroach.roachpb.Internal/Batch | 29 |
| 2018-06-04 19:47:09.798587+00:00 | 10ms530µs114ns | === SPAN START: dist sender === | | | dist sender | 30 |
| 2018-06-04 19:47:09.798593+00:00 | 10ms536µs498ns | querying next range at /Table/52/1/800 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 30 |
| 2018-06-04 19:47:09.798606+00:00 | 10ms548µs907ns | r32: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 30 |
| 2018-06-04 19:47:09.798617+00:00 | 10ms560µs452ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 30 |
| 2018-06-04 19:47:09.798624+00:00 | 10ms567µs179ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 31 |
| 2018-06-04 19:47:09.798626+00:00 | 10ms569µs543ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 31 |
| 2018-06-04 19:47:09.798629+00:00 | 10ms572µs177ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 31 |
| 2018-06-04 19:47:09.798633+00:00 | 10ms576µs390ns | read-only path | [n1,s1,r32/1:/Table/52/1/{800-900}] | | /cockroach.roachpb.Internal/Batch | 31 |
| 2018-06-04 19:47:09.798649+00:00 | 10ms592µs504ns | replica.Send got error: [NotLeaseHolderError] r32: replica | [n1,s1,r32/1:/Table/52/1/{800-900}] | | /cockroach.roachpb.Internal/Batch | 31 |
| | | (n1,s1):1 not lease holder; current lease is repl=(n3,s3):2 seq=5 | | | | |
| | | start=1528141626.112756917,0 epo=2 pro=1528141626.112760140,0 | | | | |
| 2018-06-04 19:47:09.798656+00:00 | 10ms599µs223ns | *roachpb.NotLeaseHolderError | [n1] | | /cockroach.roachpb.Internal/Batch | 31 |
| 2018-06-04 19:47:09.798668+00:00 | 10ms611µs240ns | application error: [NotLeaseHolderError] r32: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 30 |
| | | not lease holder; current lease is repl=(n3,s3):2 seq=5 | | | | |
| | | start=1528141626.112756917,0 epo=2 pro=1528141626.112760140,0 | | | | |
| 2018-06-04 19:47:09.798674+00:00 | 10ms617µs285ns | error: {(err: [NotLeaseHolderError] r32: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 30 |
| | | not lease holder; current lease is repl=(n3,s3):2 seq=5 | | | | |
| | | start=1528141626.112756917,0 epo=2 pro=1528141626.112760140,0) <nil>}; | | | | |
| | | trying next peer (n3,s3):2 | | | | |
| 2018-06-04 19:47:09.798685+00:00 | 10ms628µs356ns | sending request to alex-laptop.local:26261 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 30 |
| 2018-06-04 19:47:09.798696+00:00 | 10ms638µs861ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 32 |
| 2018-06-04 19:47:09.798848+00:00 | 10ms791µs559ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 404 |
| 2018-06-04 19:47:09.798876+00:00 | 10ms819µs49ns | 1 LeaseInfo | [n3] | | /cockroach.roachpb.Internal/Batch | 404 |
| 2018-06-04 19:47:09.798881+00:00 | 10ms824µs205ns | executing 1 requests | [n3,s3] | | /cockroach.roachpb.Internal/Batch | 404 |
| 2018-06-04 19:47:09.798887+00:00 | 10ms830µs585ns | read-only path | [n3,s3,r32/2:/Table/52/1/{800-900}] | | /cockroach.roachpb.Internal/Batch | 404 |
| 2018-06-04 19:47:09.798893+00:00 | 10ms836µs689ns | command queue | [n3,s3,r32/2:/Table/52/1/{800-900}] | | /cockroach.roachpb.Internal/Batch | 404 |
| 2018-06-04 19:47:09.798897+00:00 | 10ms840µs441ns | waiting for read lock | [n3,s3,r32/2:/Table/52/1/{800-900}] | | /cockroach.roachpb.Internal/Batch | 404 |
| 2018-06-04 19:47:09.798905+00:00 | 10ms848µs361ns | read completed | [n3,s3,r32/2:/Table/52/1/{800-900}] | | /cockroach.roachpb.Internal/Batch | 404 |
| 2018-06-04 19:47:09.799081+00:00 | 11ms24µs729ns | === SPAN START: dist sender === | | | dist sender | 33 |
| 2018-06-04 19:47:09.799091+00:00 | 11ms33µs827ns | querying next range at /Table/52/1/900 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 33 |
| 2018-06-04 19:47:09.799105+00:00 | 11ms48µs744ns | r33: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 33 |
| 2018-06-04 19:47:09.799119+00:00 | 11ms62µs232ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 33 |
| 2018-06-04 19:47:09.799127+00:00 | 11ms70µs419ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 34 |
| 2018-06-04 19:47:09.79913+00:00 | 11ms72µs876ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 34 |
| 2018-06-04 19:47:09.799133+00:00 | 11ms76µs95ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 34 |
| 2018-06-04 19:47:09.799137+00:00 | 11ms80µs463ns | read-only path | [n1,s1,r33/1:/Table/52/1/{900-1000}] | | /cockroach.roachpb.Internal/Batch | 34 |
| 2018-06-04 19:47:09.799141+00:00 | 11ms84µs174ns | read has no clock uncertainty | [n1,s1,r33/1:/Table/52/1/{900-1000}] | | /cockroach.roachpb.Internal/Batch | 34 |
| 2018-06-04 19:47:09.799143+00:00 | 11ms86µs331ns | command queue | [n1,s1,r33/1:/Table/52/1/{900-1000}] | | /cockroach.roachpb.Internal/Batch | 34 |
| 2018-06-04 19:47:09.799147+00:00 | 11ms89µs818ns | waiting for read lock | [n1,s1,r33/1:/Table/52/1/{900-1000}] | | /cockroach.roachpb.Internal/Batch | 34 |
| 2018-06-04 19:47:09.799153+00:00 | 11ms96µs4ns | read completed | [n1,s1,r33/1:/Table/52/1/{900-1000}] | | /cockroach.roachpb.Internal/Batch | 34 |
| 2018-06-04 19:47:09.799181+00:00 | 11ms124µs803ns | === SPAN START: dist sender === | | | dist sender | 35 |
| 2018-06-04 19:47:09.799188+00:00 | 11ms131µs513ns | querying next range at /Table/52/1/1000 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 35 |
| 2018-06-04 19:47:09.799201+00:00 | 11ms144µs41ns | r34: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 35 |
| 2018-06-04 19:47:09.799212+00:00 | 11ms155µs730ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 35 |
| 2018-06-04 19:47:09.799219+00:00 | 11ms162µs489ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 36 |
| 2018-06-04 19:47:09.799222+00:00 | 11ms164µs925ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 36 |
| 2018-06-04 19:47:09.799224+00:00 | 11ms167µs586ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 36 |
| 2018-06-04 19:47:09.799228+00:00 | 11ms171µs190ns | read-only path | [n1,s1,r34/1:/Table/52/1/1{000-100}] | | /cockroach.roachpb.Internal/Batch | 36 |
| 2018-06-04 19:47:09.799231+00:00 | 11ms174µs251ns | read has no clock uncertainty | [n1,s1,r34/1:/Table/52/1/1{000-100}] | | /cockroach.roachpb.Internal/Batch | 36 |
| 2018-06-04 19:47:09.799233+00:00 | 11ms176µs113ns | command queue | [n1,s1,r34/1:/Table/52/1/1{000-100}] | | /cockroach.roachpb.Internal/Batch | 36 |
| 2018-06-04 19:47:09.799236+00:00 | 11ms179µs378ns | waiting for read lock | [n1,s1,r34/1:/Table/52/1/1{000-100}] | | /cockroach.roachpb.Internal/Batch | 36 |
| 2018-06-04 19:47:09.799241+00:00 | 11ms184µs431ns | read completed | [n1,s1,r34/1:/Table/52/1/1{000-100}] | | /cockroach.roachpb.Internal/Batch | 36 |
| 2018-06-04 19:47:09.799268+00:00 | 11ms211µs104ns | === SPAN START: dist sender === | | | dist sender | 37 |
| 2018-06-04 19:47:09.799274+00:00 | 11ms217µs289ns | querying next range at /Table/52/1/1100 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 37 |
| 2018-06-04 19:47:09.799285+00:00 | 11ms228µs601ns | r35: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 37 |
| 2018-06-04 19:47:09.799297+00:00 | 11ms239µs819ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 37 |
| 2018-06-04 19:47:09.799304+00:00 | 11ms246µs835ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 38 |
| 2018-06-04 19:47:09.799306+00:00 | 11ms249µs97ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 38 |
| 2018-06-04 19:47:09.799308+00:00 | 11ms251µs584ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 38 |
| 2018-06-04 19:47:09.799313+00:00 | 11ms256µs127ns | read-only path | [n1,s1,r35/1:/Table/52/1/1{100-200}] | | /cockroach.roachpb.Internal/Batch | 38 |
| 2018-06-04 19:47:09.799316+00:00 | 11ms258µs950ns | read has no clock uncertainty | [n1,s1,r35/1:/Table/52/1/1{100-200}] | | /cockroach.roachpb.Internal/Batch | 38 |
| 2018-06-04 19:47:09.799318+00:00 | 11ms260µs864ns | command queue | [n1,s1,r35/1:/Table/52/1/1{100-200}] | | /cockroach.roachpb.Internal/Batch | 38 |
| 2018-06-04 19:47:09.799321+00:00 | 11ms263µs931ns | waiting for read lock | [n1,s1,r35/1:/Table/52/1/1{100-200}] | | /cockroach.roachpb.Internal/Batch | 38 |
| 2018-06-04 19:47:09.799326+00:00 | 11ms268µs962ns | read completed | [n1,s1,r35/1:/Table/52/1/1{100-200}] | | /cockroach.roachpb.Internal/Batch | 38 |
| 2018-06-04 19:47:09.799349+00:00 | 11ms292µs614ns | === SPAN START: dist sender === | | | dist sender | 39 |
| 2018-06-04 19:47:09.799355+00:00 | 11ms298µs548ns | querying next range at /Table/52/1/1200 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 39 |
| 2018-06-04 19:47:09.799367+00:00 | 11ms309µs838ns | r36: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 39 |
| 2018-06-04 19:47:09.799377+00:00 | 11ms320µs560ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 39 |
| 2018-06-04 19:47:09.799384+00:00 | 11ms327µs77ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 40 |
| 2018-06-04 19:47:09.799386+00:00 | 11ms329µs120ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 40 |
| 2018-06-04 19:47:09.799388+00:00 | 11ms331µs549ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 40 |
| 2018-06-04 19:47:09.799392+00:00 | 11ms335µs272ns | read-only path | [n1,s1,r36/1:/Table/52/1/1{200-300}] | | /cockroach.roachpb.Internal/Batch | 40 |
| 2018-06-04 19:47:09.799395+00:00 | 11ms337µs823ns | read has no clock uncertainty | [n1,s1,r36/1:/Table/52/1/1{200-300}] | | /cockroach.roachpb.Internal/Batch | 40 |
| 2018-06-04 19:47:09.799396+00:00 | 11ms339µs672ns | command queue | [n1,s1,r36/1:/Table/52/1/1{200-300}] | | /cockroach.roachpb.Internal/Batch | 40 |
| 2018-06-04 19:47:09.7994+00:00 | 11ms343µs370ns | waiting for read lock | [n1,s1,r36/1:/Table/52/1/1{200-300}] | | /cockroach.roachpb.Internal/Batch | 40 |
| 2018-06-04 19:47:09.799406+00:00 | 11ms349µs664ns | read completed | [n1,s1,r36/1:/Table/52/1/1{200-300}] | | /cockroach.roachpb.Internal/Batch | 40 |
| 2018-06-04 19:47:09.79943+00:00 | 11ms373µs299ns | === SPAN START: dist sender === | | | dist sender | 41 |
| 2018-06-04 19:47:09.799436+00:00 | 11ms379µs232ns | querying next range at /Table/52/1/1300 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 41 |
| 2018-06-04 19:47:09.799447+00:00 | 11ms389µs938ns | r37: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 41 |
| 2018-06-04 19:47:09.799457+00:00 | 11ms400µs382ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 41 |
| 2018-06-04 19:47:09.799463+00:00 | 11ms406µs760ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 42 |
| 2018-06-04 19:47:09.799466+00:00 | 11ms408µs944ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 42 |
| 2018-06-04 19:47:09.799468+00:00 | 11ms411µs228ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 42 |
| 2018-06-04 19:47:09.799471+00:00 | 11ms414µs681ns | read-only path | [n1,s1,r37/1:/Table/52/1/1{300-400}] | | /cockroach.roachpb.Internal/Batch | 42 |
| 2018-06-04 19:47:09.799474+00:00 | 11ms417µs409ns | read has no clock uncertainty | [n1,s1,r37/1:/Table/52/1/1{300-400}] | | /cockroach.roachpb.Internal/Batch | 42 |
| 2018-06-04 19:47:09.799476+00:00 | 11ms419µs146ns | command queue | [n1,s1,r37/1:/Table/52/1/1{300-400}] | | /cockroach.roachpb.Internal/Batch | 42 |
| 2018-06-04 19:47:09.799479+00:00 | 11ms421µs860ns | waiting for read lock | [n1,s1,r37/1:/Table/52/1/1{300-400}] | | /cockroach.roachpb.Internal/Batch | 42 |
| 2018-06-04 19:47:09.799484+00:00 | 11ms427µs16ns | read completed | [n1,s1,r37/1:/Table/52/1/1{300-400}] | | /cockroach.roachpb.Internal/Batch | 42 |
| 2018-06-04 19:47:09.799511+00:00 | 11ms454µs779ns | === SPAN START: dist sender === | | | dist sender | 43 |
| 2018-06-04 19:47:09.799518+00:00 | 11ms461µs141ns | querying next range at /Table/52/1/1400 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 43 |
| 2018-06-04 19:47:09.799529+00:00 | 11ms472µs389ns | r38: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 43 |
| 2018-06-04 19:47:09.79954+00:00 | 11ms482µs907ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 43 |
| 2018-06-04 19:47:09.799546+00:00 | 11ms489µs232ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 44 |
| 2018-06-04 19:47:09.799548+00:00 | 11ms491µs269ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 44 |
| 2018-06-04 19:47:09.79955+00:00 | 11ms493µs562ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 44 |
| 2018-06-04 19:47:09.799554+00:00 | 11ms497µs691ns | read-only path | [n1,s1,r38/1:/Table/52/1/1{400-500}] | | /cockroach.roachpb.Internal/Batch | 44 |
| 2018-06-04 19:47:09.799557+00:00 | 11ms500µs623ns | read has no clock uncertainty | [n1,s1,r38/1:/Table/52/1/1{400-500}] | | /cockroach.roachpb.Internal/Batch | 44 |
| 2018-06-04 19:47:09.799559+00:00 | 11ms502µs327ns | command queue | [n1,s1,r38/1:/Table/52/1/1{400-500}] | | /cockroach.roachpb.Internal/Batch | 44 |
| 2018-06-04 19:47:09.799562+00:00 | 11ms505µs180ns | waiting for read lock | [n1,s1,r38/1:/Table/52/1/1{400-500}] | | /cockroach.roachpb.Internal/Batch | 44 |
| 2018-06-04 19:47:09.799567+00:00 | 11ms509µs994ns | read completed | [n1,s1,r38/1:/Table/52/1/1{400-500}] | | /cockroach.roachpb.Internal/Batch | 44 |
| 2018-06-04 19:47:09.79959+00:00 | 11ms532µs820ns | === SPAN START: dist sender === | | | dist sender | 45 |
| 2018-06-04 19:47:09.799595+00:00 | 11ms538µs776ns | querying next range at /Table/52/1/1500 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 45 |
| 2018-06-04 19:47:09.799606+00:00 | 11ms548µs902ns | r39: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 45 |
| 2018-06-04 19:47:09.799617+00:00 | 11ms560µs329ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 45 |
| 2018-06-04 19:47:09.799623+00:00 | 11ms566µs670ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 46 |
| 2018-06-04 19:47:09.799626+00:00 | 11ms568µs874ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 46 |
| 2018-06-04 19:47:09.799628+00:00 | 11ms571µs211ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 46 |
| 2018-06-04 19:47:09.799631+00:00 | 11ms574µs749ns | read-only path | [n1,s1,r39/1:/Table/52/1/1{500-600}] | | /cockroach.roachpb.Internal/Batch | 46 |
| 2018-06-04 19:47:09.799635+00:00 | 11ms578µs106ns | request range lease (attempt #1) | [n1,s1,r39/1:/Table/52/1/1{500-600}] | | /cockroach.roachpb.Internal/Batch | 46 |
| 2018-06-04 19:47:09.799639+00:00 | 11ms582µs737ns | === SPAN START: request range lease === | | | request range lease | 47 |
| 2018-06-04 19:47:09.799644+00:00 | 11ms587µs446ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 48 |
| | | === | | | | |
| 2018-06-04 19:47:09.799664+00:00 | 11ms606µs865ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 49 |
| 2018-06-04 19:47:09.79974+00:00 | 11ms683µs198ns | === SPAN START: dist sender === | | | dist sender | 50 |
| 2018-06-04 19:47:09.799756+00:00 | 11ms698µs819ns | querying next range at /System/NodeLiveness/1 | [s1,r39/1:/Table/52/1/1{500-600},txn=c4d396f8,n1] | | dist sender | 50 |
| 2018-06-04 19:47:09.799779+00:00 | 11ms722µs748ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r39/1:/Table/52/1/1{500-600},txn=c4d396f8,n1] | | dist sender | 50 |
| 2018-06-04 19:47:09.799797+00:00 | 11ms739µs976ns | sending request to alex-laptop.local:26259 | [s1,r39/1:/Table/52/1/1{500-600},txn=c4d396f8,n1] | | dist sender | 50 |
| 2018-06-04 19:47:09.799808+00:00 | 11ms751µs751ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 51 |
| 2018-06-04 19:47:09.800025+00:00 | 11ms968µs83ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 405 |
| 2018-06-04 19:47:09.800045+00:00 | 11ms987µs998ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 405 |
| 2018-06-04 19:47:09.800051+00:00 | 11ms993µs935ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 405 |
| 2018-06-04 19:47:09.800057+00:00 | 12ms336ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 405 |
| 2018-06-04 19:47:09.800071+00:00 | 12ms14µs577ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 405 |
| 2018-06-04 19:47:09.800089+00:00 | 12ms32µs342ns | === SPAN START: request range lease === | | | request range lease | 406 |
| 2018-06-04 19:47:09.800096+00:00 | 12ms38µs832ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 407 |
| | | === | | | | |
| 2018-06-04 19:47:09.800118+00:00 | 12ms60µs911ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 405 |
| 2018-06-04 19:47:09.800133+00:00 | 12ms76µs336ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | [async] storage.pendingLeaseRequest: requesting lease | 407 |
| 2018-06-04 19:47:09.800146+00:00 | 12ms89µs501ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | [async] storage.pendingLeaseRequest: requesting lease | 407 |
| 2018-06-04 19:47:09.800197+00:00 | 12ms139µs827ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 405 |
| 2018-06-04 19:47:09.800201+00:00 | 12ms143µs876ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 405 |
| 2018-06-04 19:47:09.80024+00:00 | 12ms183µs91ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | [async] storage.pendingLeaseRequest: requesting lease | 407 |
| 2018-06-04 19:47:09.800243+00:00 | 12ms186µs633ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | [async] storage.pendingLeaseRequest: requesting lease | 407 |
| 2018-06-04 19:47:09.800877+00:00 | 12ms820µs320ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 405 |
| 2018-06-04 19:47:09.801195+00:00 | 13ms138µs200ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 405 |
| 2018-06-04 19:47:09.801217+00:00 | 13ms160µs744ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 405 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:47:09.801272+00:00 | 13ms214µs980ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | [async] storage.pendingLeaseRequest: requesting lease | 407 |
| 2018-06-04 19:47:09.801681+00:00 | 13ms624µs430ns | heartbeat 1528141639.299667112,0 | [n1,s1,r39/1:/Table/52/1/1{500-600}] | | liveness heartbeat | 49 |
| 2018-06-04 19:47:09.801691+00:00 | 13ms634µs18ns | read-write path | [n1,s1,r39/1:/Table/52/1/1{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 48 |
| 2018-06-04 19:47:09.801699+00:00 | 13ms642µs511ns | applied timestamp cache | [n1,s1,r39/1:/Table/52/1/1{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 48 |
| 2018-06-04 19:47:09.801786+00:00 | 13ms729µs92ns | evaluated request | [n1,s1,r39/1:/Table/52/1/1{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 48 |
| 2018-06-04 19:47:09.801794+00:00 | 13ms736µs990ns | acquired {raft,replica}mu | [n1,s1,r39/1:/Table/52/1/1{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 48 |
| 2018-06-04 19:47:09.802375+00:00 | 14ms317µs824ns | applying command | [n1,s1,r39/1:/Table/52/1/1{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 48 |
| 2018-06-04 19:47:09.802458+00:00 | 14ms401µs329ns | new range lease repl=(n1,s1):1 seq=4 start=1528141568.365104551,0 | [n1,s1,r39/1:/Table/52/1/1{500-600}] | storage/replica_proposal.go:202 | [async] storage.pendingLeaseRequest: requesting lease | 48 |
| | | epo=3 pro=1528141629.799637067,0 following repl=(n1,s1):1 seq=3 | | | | |
| | | start=1528141259.602787924,0 epo=1 pro=1528141365.657033828,0 | | | | |
| 2018-06-04 19:47:09.802555+00:00 | 14ms498µs482ns | === SPAN START: [async] storage.Store: gossip on capacity change === | | | [async] storage.Store: gossip on capacity change | 52 |
| 2018-06-04 19:47:09.80257+00:00 | 14ms513µs274ns | not gossiping system config because the replica doesn't contain the | [n1,s1,r39/1:/Table/52/1/1{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 48 |
| | | system config's start key | | | | |
| 2018-06-04 19:47:09.802751+00:00 | 14ms694µs592ns | lease acquisition succeeded: repl=(n1,s1):1 seq=3 | [n1,s1,r39/1:/Table/52/1/1{500-600}] | | /cockroach.roachpb.Internal/Batch | 46 |
| | | start=1528141259.602787924,0 epo=1 pro=1528141365.657033828,0 | | | | |
| 2018-06-04 19:47:09.802758+00:00 | 14ms700µs902ns | read has no clock uncertainty | [n1,s1,r39/1:/Table/52/1/1{500-600}] | | /cockroach.roachpb.Internal/Batch | 46 |
| 2018-06-04 19:47:09.802761+00:00 | 14ms704µs749ns | command queue | [n1,s1,r39/1:/Table/52/1/1{500-600}] | | /cockroach.roachpb.Internal/Batch | 46 |
| 2018-06-04 19:47:09.802766+00:00 | 14ms709µs244ns | waiting for read lock | [n1,s1,r39/1:/Table/52/1/1{500-600}] | | /cockroach.roachpb.Internal/Batch | 46 |
| 2018-06-04 19:47:09.802775+00:00 | 14ms718µs42ns | read completed | [n1,s1,r39/1:/Table/52/1/1{500-600}] | | /cockroach.roachpb.Internal/Batch | 46 |
| 2018-06-04 19:47:09.802818+00:00 | 14ms761µs237ns | === SPAN START: dist sender === | | | dist sender | 53 |
| 2018-06-04 19:47:09.802827+00:00 | 14ms770µs551ns | querying next range at /Table/52/1/1600 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 53 |
| 2018-06-04 19:47:09.802845+00:00 | 14ms788µs722ns | r40: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 53 |
| 2018-06-04 19:47:09.802857+00:00 | 14ms800µs30ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 53 |
| 2018-06-04 19:47:09.802866+00:00 | 14ms809µs157ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 54 |
| 2018-06-04 19:47:09.802868+00:00 | 14ms811µs383ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 54 |
| 2018-06-04 19:47:09.802873+00:00 | 14ms816µs327ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 54 |
| 2018-06-04 19:47:09.802944+00:00 | 14ms887µs715ns | read-only path | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | /cockroach.roachpb.Internal/Batch | 54 |
| 2018-06-04 19:47:09.802948+00:00 | 14ms891µs505ns | request range lease (attempt #1) | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | /cockroach.roachpb.Internal/Batch | 54 |
| 2018-06-04 19:47:09.802953+00:00 | 14ms896µs507ns | === SPAN START: request range lease === | | | request range lease | 55 |
| 2018-06-04 19:47:09.802959+00:00 | 14ms902µs784ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 56 |
| | | === | | | | |
| 2018-06-04 19:47:09.802979+00:00 | 14ms922µs158ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 57 |
| 2018-06-04 19:47:09.803037+00:00 | 14ms980µs256ns | === SPAN START: dist sender === | | | dist sender | 58 |
| 2018-06-04 19:47:09.803046+00:00 | 14ms989µs734ns | querying next range at /System/NodeLiveness/1 | [s1,r40/1:/Table/52/1/1{600-700},txn=fbaf8b93,n1] | | dist sender | 58 |
| 2018-06-04 19:47:09.803064+00:00 | 15ms6µs909ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r40/1:/Table/52/1/1{600-700},txn=fbaf8b93,n1] | | dist sender | 58 |
| 2018-06-04 19:47:09.803077+00:00 | 15ms20µs643ns | sending request to alex-laptop.local:26259 | [s1,r40/1:/Table/52/1/1{600-700},txn=fbaf8b93,n1] | | dist sender | 58 |
| 2018-06-04 19:47:09.803091+00:00 | 15ms34µs745ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 59 |
| 2018-06-04 19:47:09.803314+00:00 | 15ms257µs415ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 408 |
| 2018-06-04 19:47:09.803335+00:00 | 15ms278µs808ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 408 |
| 2018-06-04 19:47:09.803341+00:00 | 15ms284µs734ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 408 |
| 2018-06-04 19:47:09.803346+00:00 | 15ms289µs624ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 408 |
| 2018-06-04 19:47:09.803356+00:00 | 15ms298µs994ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 408 |
| 2018-06-04 19:47:09.803376+00:00 | 15ms318µs859ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 408 |
| 2018-06-04 19:47:09.803434+00:00 | 15ms377µs359ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 408 |
| 2018-06-04 19:47:09.803438+00:00 | 15ms380µs920ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 408 |
| 2018-06-04 19:47:09.804008+00:00 | 15ms951µs669ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 408 |
| 2018-06-04 19:47:09.804247+00:00 | 16ms190µs5ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 408 |
| 2018-06-04 19:47:09.804264+00:00 | 16ms207µs194ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 408 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:47:09.804702+00:00 | 16ms644µs994ns | heartbeat 1528141639.302982820,0 | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | liveness heartbeat | 57 |
| 2018-06-04 19:47:09.804708+00:00 | 16ms651µs26ns | read-write path | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 56 |
| 2018-06-04 19:47:09.804714+00:00 | 16ms657µs27ns | applied timestamp cache | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 56 |
| 2018-06-04 19:47:09.804777+00:00 | 16ms720µs664ns | evaluated request | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 56 |
| 2018-06-04 19:47:09.80478+00:00 | 16ms723µs77ns | acquired {raft,replica}mu | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 56 |
| 2018-06-04 19:47:12.779563+00:00 | 2s991ms506µs112ns | retry proposal 2c6ad45035f163db: reasonTicks | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 56 |
| 2018-06-04 19:47:12.779753+00:00 | 2s991ms696µs54ns | applied timestamp cache | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 56 |
| 2018-06-04 19:47:12.779911+00:00 | 2s991ms854µs220ns | evaluated request | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 56 |
| 2018-06-04 19:47:12.779916+00:00 | 2s991ms859µs265ns | acquired {raft,replica}mu | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 56 |
| 2018-06-04 19:47:13.378666+00:00 | 3s590ms609µs485ns | retry proposal 582601a7fd4ad862: reasonNewLeader | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 56 |
| 2018-06-04 19:47:13.378741+00:00 | 3s590ms684µs549ns | applied timestamp cache | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 56 |
| 2018-06-04 19:47:13.379324+00:00 | 3s591ms267µs602ns | evaluated request | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 56 |
| 2018-06-04 19:47:13.379477+00:00 | 3s591ms420µs344ns | acquired {raft,replica}mu | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 56 |
| 2018-06-04 19:47:13.380982+00:00 | 3s592ms925µs348ns | applying command | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 56 |
| 2018-06-04 19:47:13.381162+00:00 | 3s593ms105µs514ns | new range lease repl=(n1,s1):1 seq=6 start=1528141568.365216808,0 | [n1,s1,r40/1:/Table/52/1/1{600-700}] | storage/replica_proposal.go:202 | [async] storage.pendingLeaseRequest: requesting lease | 56 |
| | | epo=3 pro=1528141629.802950022,0 following repl=(n1,s1):1 seq=5 | | | | |
| | | start=1528141460.337622006,0 epo=1 pro=1528141460.337627515,0 | | | | |
| 2018-06-04 19:47:13.381275+00:00 | 3s593ms218µs570ns | === SPAN START: [async] storage.Store: gossip on capacity change === | | | [async] storage.Store: gossip on capacity change | 60 |
| 2018-06-04 19:47:13.381301+00:00 | 3s593ms244µs752ns | not gossiping system config because the replica doesn't contain the | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 56 |
| | | system config's start key | | | | |
| 2018-06-04 19:47:13.381557+00:00 | 3s593ms500µs372ns | lease acquisition succeeded: repl=(n1,s1):1 seq=5 | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | /cockroach.roachpb.Internal/Batch | 54 |
| | | start=1528141460.337622006,0 epo=1 pro=1528141460.337627515,0 | | | | |
| 2018-06-04 19:47:13.381573+00:00 | 3s593ms516µs107ns | read has no clock uncertainty | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | /cockroach.roachpb.Internal/Batch | 54 |
| 2018-06-04 19:47:13.381583+00:00 | 3s593ms526µs270ns | command queue | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | /cockroach.roachpb.Internal/Batch | 54 |
| 2018-06-04 19:47:13.381604+00:00 | 3s593ms547µs477ns | waiting for read lock | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | /cockroach.roachpb.Internal/Batch | 54 |
| 2018-06-04 19:47:13.381642+00:00 | 3s593ms585µs645ns | read completed | [n1,s1,r40/1:/Table/52/1/1{600-700}] | | /cockroach.roachpb.Internal/Batch | 54 |
| 2018-06-04 19:47:13.381844+00:00 | 3s593ms787µs303ns | === SPAN START: dist sender === | | | dist sender | 61 |
| 2018-06-04 19:47:13.381877+00:00 | 3s593ms820µs536ns | querying next range at /Table/52/1/1700 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 61 |
| 2018-06-04 19:47:13.381918+00:00 | 3s593ms861µs756ns | === SPAN START: range lookup === | | | range lookup | 62 |
| 2018-06-04 19:47:13.381955+00:00 | 3s593ms898µs450ns | === SPAN START: dist sender === | | | dist sender | 63 |
| 2018-06-04 19:47:13.382059+00:00 | 3s594ms2µs277ns | querying next range at /Meta2/Table/52/1/1700/NULL | [client=[::1]:57501,user=root,txn=9352587e,range-lookup=/Table/52/1/1700,n1] | | dist sender | 63 |
| 2018-06-04 19:47:13.382189+00:00 | 3s594ms131µs853ns | r1: sending batch 1 Scan to (n2,s2):3 | [client=[::1]:57501,user=root,txn=9352587e,range-lookup=/Table/52/1/1700,n1] | | dist sender | 63 |
| 2018-06-04 19:47:13.382245+00:00 | 3s594ms188µs149ns | sending request to alex-laptop.local:26259 | [client=[::1]:57501,user=root,txn=9352587e,range-lookup=/Table/52/1/1700,n1] | | dist sender | 63 |
| 2018-06-04 19:47:13.382256+00:00 | 3s594ms199µs703ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 64 |
| 2018-06-04 19:47:13.383479+00:00 | 3s595ms422µs379ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 409 |
| 2018-06-04 19:47:13.383569+00:00 | 3s595ms512µs463ns | 1 Scan | [n2] | | /cockroach.roachpb.Internal/Batch | 409 |
| 2018-06-04 19:47:13.383594+00:00 | 3s595ms537µs413ns | executing 1 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 409 |
| 2018-06-04 19:47:13.383655+00:00 | 3s595ms597µs945ns | read-only path | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 409 |
| 2018-06-04 19:47:13.383693+00:00 | 3s595ms635µs985ns | command queue | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 409 |
| 2018-06-04 19:47:13.383696+00:00 | 3s595ms639µs678ns | operation accepts inconsistent results | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 409 |
| 2018-06-04 19:47:13.383708+00:00 | 3s595ms651µs671ns | waiting for read lock | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 409 |
| 2018-06-04 19:47:13.384648+00:00 | 3s596ms590µs976ns | engine stats: {InternalDeleteSkippedCount:10 TimeBoundNumSSTs:0} | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 409 |
| 2018-06-04 19:47:13.384719+00:00 | 3s596ms662µs806ns | read completed | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 409 |
| 2018-06-04 19:47:13.385399+00:00 | 3s597ms342µs233ns | looked up range descriptor | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 61 |
| 2018-06-04 19:47:13.385444+00:00 | 3s597ms387µs239ns | r41: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 61 |
| 2018-06-04 19:47:13.385473+00:00 | 3s597ms416µs484ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 61 |
| 2018-06-04 19:47:13.38549+00:00 | 3s597ms433µs342ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 65 |
| 2018-06-04 19:47:13.385498+00:00 | 3s597ms441µs120ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 65 |
| 2018-06-04 19:47:13.385507+00:00 | 3s597ms449µs940ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 65 |
| 2018-06-04 19:47:13.385673+00:00 | 3s597ms616µs462ns | read-only path | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | /cockroach.roachpb.Internal/Batch | 65 |
| 2018-06-04 19:47:13.385681+00:00 | 3s597ms623µs976ns | request range lease (attempt #1) | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | /cockroach.roachpb.Internal/Batch | 65 |
| 2018-06-04 19:47:13.38569+00:00 | 3s597ms633µs245ns | === SPAN START: request range lease === | | | request range lease | 66 |
| 2018-06-04 19:47:13.3857+00:00 | 3s597ms643µs99ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 67 |
| | | === | | | | |
| 2018-06-04 19:47:13.385726+00:00 | 3s597ms669µs538ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 68 |
| 2018-06-04 19:47:13.385828+00:00 | 3s597ms771µs471ns | === SPAN START: dist sender === | | | dist sender | 69 |
| 2018-06-04 19:47:13.385849+00:00 | 3s597ms792µs213ns | querying next range at /System/NodeLiveness/1 | [s1,r41/1:/Table/52/1/1{700-800},txn=aeef0d78,n1] | | dist sender | 69 |
| 2018-06-04 19:47:13.385879+00:00 | 3s597ms822µs67ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r41/1:/Table/52/1/1{700-800},txn=aeef0d78,n1] | | dist sender | 69 |
| 2018-06-04 19:47:13.3859+00:00 | 3s597ms843µs213ns | sending request to alex-laptop.local:26259 | [s1,r41/1:/Table/52/1/1{700-800},txn=aeef0d78,n1] | | dist sender | 69 |
| 2018-06-04 19:47:13.385915+00:00 | 3s597ms858µs285ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 70 |
| 2018-06-04 19:47:13.386305+00:00 | 3s598ms248µs434ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 410 |
| 2018-06-04 19:47:13.386357+00:00 | 3s598ms300µs293ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 410 |
| 2018-06-04 19:47:13.386373+00:00 | 3s598ms316µs656ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 410 |
| 2018-06-04 19:47:13.386388+00:00 | 3s598ms331µs451ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 410 |
| 2018-06-04 19:47:13.386418+00:00 | 3s598ms361µs366ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 410 |
| 2018-06-04 19:47:13.386489+00:00 | 3s598ms432µs646ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 410 |
| 2018-06-04 19:47:13.386621+00:00 | 3s598ms564µs191ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 410 |
| 2018-06-04 19:47:13.386629+00:00 | 3s598ms572µs357ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 410 |
| 2018-06-04 19:47:13.387739+00:00 | 3s599ms682µs275ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 410 |
| 2018-06-04 19:47:13.388183+00:00 | 3s600ms125µs994ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 410 |
| 2018-06-04 19:47:13.388237+00:00 | 3s600ms180µs2ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 410 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:47:13.389449+00:00 | 3s601ms392µs381ns | heartbeat 1528141642.885732004,0 | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | liveness heartbeat | 68 |
| 2018-06-04 19:47:13.389479+00:00 | 3s601ms422µs462ns | read-write path | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 67 |
| 2018-06-04 19:47:13.389507+00:00 | 3s601ms450µs111ns | applied timestamp cache | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 67 |
| 2018-06-04 19:47:13.389815+00:00 | 3s601ms758µs530ns | evaluated request | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 67 |
| 2018-06-04 19:47:13.389826+00:00 | 3s601ms769µs139ns | acquired {raft,replica}mu | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 67 |
| 2018-06-04 19:47:16.378799+00:00 | 6s590ms742µs654ns | retry proposal 50397e81a2bd7e83: reasonTicks | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 67 |
| 2018-06-04 19:47:16.378956+00:00 | 6s590ms899µs155ns | applied timestamp cache | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 67 |
| 2018-06-04 19:47:16.379316+00:00 | 6s591ms259µs198ns | evaluated request | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 67 |
| 2018-06-04 19:47:16.379326+00:00 | 6s591ms269µs722ns | acquired {raft,replica}mu | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 67 |
| 2018-06-04 19:47:16.581442+00:00 | 6s793ms384µs914ns | retry proposal 52d17b0bb61814c0: reasonNewLeader | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 67 |
| 2018-06-04 19:47:16.581726+00:00 | 6s793ms669µs21ns | applied timestamp cache | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 67 |
| 2018-06-04 19:47:16.583367+00:00 | 6s795ms310µs768ns | evaluated request | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 67 |
| 2018-06-04 19:47:16.583413+00:00 | 6s795ms355µs946ns | acquired {raft,replica}mu | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 67 |
| 2018-06-04 19:47:16.585453+00:00 | 6s797ms395µs860ns | retry proposal 7ce5ca9e7298e9d4: reasonNewLeaderOrConfigChange | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 67 |
| 2018-06-04 19:47:16.585601+00:00 | 6s797ms544µs811ns | applied timestamp cache | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 67 |
| 2018-06-04 19:47:16.58639+00:00 | 6s798ms333µs721ns | evaluated request | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 67 |
| 2018-06-04 19:47:16.586431+00:00 | 6s798ms373µs989ns | acquired {raft,replica}mu | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 67 |
| 2018-06-04 19:47:16.589585+00:00 | 6s801ms528µs468ns | command proposed from replica (n1,s1):1 with lease #3 incompatible | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 67 |
| | | to repl=(n1,s1):1 seq=4 start=1528141568.365333322,0 epo=3 | | | | |
| | | pro=1528141633.385683914,0 | | | | |
| 2018-06-04 19:47:16.58965+00:00 | 6s801ms593µs206ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 67 |
| | | seq=4 start=1528141568.365333322,0 epo=3 pro=1528141633.385683914,0 | | | | |
| | | with repl=(n1,s1):1 seq=4 start=1528141568.365333322,0 epo=3 | | | | |
| | | pro=1528141633.385683914,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:16.590403+00:00 | 6s802ms346µs692ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=4 | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 67 |
| | | start=1528141568.365333322,0 epo=3 pro=1528141633.385683914,0 | | | | |
| | | with repl=(n1,s1):1 seq=4 start=1528141568.365333322,0 epo=3 | | | | |
| | | pro=1528141633.385683914,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:16.590588+00:00 | 6s802ms530µs888ns | read has no clock uncertainty | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | /cockroach.roachpb.Internal/Batch | 65 |
| 2018-06-04 19:47:16.590613+00:00 | 6s802ms556µs777ns | command queue | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | /cockroach.roachpb.Internal/Batch | 65 |
| 2018-06-04 19:47:16.590658+00:00 | 6s802ms600µs991ns | waiting for read lock | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | /cockroach.roachpb.Internal/Batch | 65 |
| 2018-06-04 19:47:16.590753+00:00 | 6s802ms696µs119ns | read completed | [n1,s1,r41/1:/Table/52/1/1{700-800}] | | /cockroach.roachpb.Internal/Batch | 65 |
| 2018-06-04 19:47:16.591566+00:00 | 6s803ms508µs834ns | === SPAN START: dist sender === | | | dist sender | 71 |
| 2018-06-04 19:47:16.591648+00:00 | 6s803ms591µs268ns | querying next range at /Table/52/1/1800 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 71 |
| 2018-06-04 19:47:16.591787+00:00 | 6s803ms730µs341ns | r42: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 71 |
| 2018-06-04 19:47:16.591883+00:00 | 6s803ms826µs588ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 71 |
| 2018-06-04 19:47:16.591939+00:00 | 6s803ms882µs729ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 72 |
| 2018-06-04 19:47:16.591958+00:00 | 6s803ms901µs4ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 72 |
| 2018-06-04 19:47:16.592042+00:00 | 6s803ms985µs442ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 72 |
| 2018-06-04 19:47:16.59259+00:00 | 6s804ms532µs974ns | read-only path | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | /cockroach.roachpb.Internal/Batch | 72 |
| 2018-06-04 19:47:16.592616+00:00 | 6s804ms559µs441ns | request range lease (attempt #1) | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | /cockroach.roachpb.Internal/Batch | 72 |
| 2018-06-04 19:47:16.592651+00:00 | 6s804ms594µs497ns | === SPAN START: request range lease === | | | request range lease | 73 |
| 2018-06-04 19:47:16.592692+00:00 | 6s804ms635µs251ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 74 |
| | | === | | | | |
| 2018-06-04 19:47:16.592749+00:00 | 6s804ms692µs518ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 75 |
| 2018-06-04 19:47:16.593069+00:00 | 6s805ms12µs297ns | === SPAN START: dist sender === | | | dist sender | 76 |
| 2018-06-04 19:47:16.593121+00:00 | 6s805ms64µs540ns | querying next range at /System/NodeLiveness/1 | [s1,r42/1:/Table/52/1/1{800-900},txn=766a1fed,n1] | | dist sender | 76 |
| 2018-06-04 19:47:16.593302+00:00 | 6s805ms245µs789ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r42/1:/Table/52/1/1{800-900},txn=766a1fed,n1] | | dist sender | 76 |
| 2018-06-04 19:47:16.593394+00:00 | 6s805ms337µs480ns | sending request to alex-laptop.local:26259 | [s1,r42/1:/Table/52/1/1{800-900},txn=766a1fed,n1] | | dist sender | 76 |
| 2018-06-04 19:47:16.593484+00:00 | 6s805ms427µs388ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 77 |
| 2018-06-04 19:47:16.594902+00:00 | 6s806ms844µs864ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 411 |
| 2018-06-04 19:47:16.594949+00:00 | 6s806ms892µs505ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 411 |
| 2018-06-04 19:47:16.594962+00:00 | 6s806ms905µs144ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 411 |
| 2018-06-04 19:47:16.594971+00:00 | 6s806ms913µs881ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 411 |
| 2018-06-04 19:47:16.594988+00:00 | 6s806ms931µs200ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 411 |
| 2018-06-04 19:47:16.595043+00:00 | 6s806ms985µs930ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 411 |
| 2018-06-04 19:47:16.595193+00:00 | 6s807ms136µs166ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 411 |
| 2018-06-04 19:47:16.595198+00:00 | 6s807ms141µs695ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 411 |
| 2018-06-04 19:47:16.596168+00:00 | 6s808ms111µs146ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 411 |
| 2018-06-04 19:47:16.596606+00:00 | 6s808ms549µs593ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 411 |
| 2018-06-04 19:47:16.596635+00:00 | 6s808ms578µs387ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 411 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:47:16.597364+00:00 | 6s809ms307µs400ns | heartbeat 1528141646.092766718,0 | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | liveness heartbeat | 75 |
| 2018-06-04 19:47:16.597374+00:00 | 6s809ms317µs238ns | read-write path | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 74 |
| 2018-06-04 19:47:16.597384+00:00 | 6s809ms327µs605ns | applied timestamp cache | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 74 |
| 2018-06-04 19:47:16.597491+00:00 | 6s809ms434µs815ns | evaluated request | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 74 |
| 2018-06-04 19:47:16.597496+00:00 | 6s809ms438µs992ns | acquired {raft,replica}mu | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 74 |
| 2018-06-04 19:47:19.576599+00:00 | 9s788ms542µs61ns | retry proposal 6cbb80367c054b66: reasonTicks | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 74 |
| 2018-06-04 19:47:19.576893+00:00 | 9s788ms836µs453ns | applied timestamp cache | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 74 |
| 2018-06-04 19:47:19.577232+00:00 | 9s789ms175µs36ns | evaluated request | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 74 |
| 2018-06-04 19:47:19.577239+00:00 | 9s789ms182µs392ns | acquired {raft,replica}mu | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 74 |
| 2018-06-04 19:47:19.977112+00:00 | 10s189ms55µs73ns | retry proposal 62b5674dc18cc9ca: reasonNewLeader | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 74 |
| 2018-06-04 19:47:19.977166+00:00 | 10s189ms109µs101ns | applied timestamp cache | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 74 |
| 2018-06-04 19:47:19.977469+00:00 | 10s189ms412µs434ns | evaluated request | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 74 |
| 2018-06-04 19:47:19.977475+00:00 | 10s189ms417µs899ns | acquired {raft,replica}mu | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 74 |
| 2018-06-04 19:47:19.977823+00:00 | 10s189ms766µs261ns | retry proposal 7c45eda8637f8bad: reasonNewLeaderOrConfigChange | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 74 |
| 2018-06-04 19:47:19.977852+00:00 | 10s189ms795µs100ns | applied timestamp cache | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 74 |
| 2018-06-04 19:47:19.977959+00:00 | 10s189ms902µs190ns | evaluated request | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 74 |
| 2018-06-04 19:47:19.977963+00:00 | 10s189ms906µs771ns | acquired {raft,replica}mu | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 74 |
| 2018-06-04 19:47:19.979633+00:00 | 10s191ms576µs171ns | command proposed from replica (n1,s1):1 with lease #5 incompatible | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 74 |
| | | to repl=(n1,s1):1 seq=6 start=1528141568.365449377,0 epo=3 | | | | |
| | | pro=1528141636.592623635,0 | | | | |
| 2018-06-04 19:47:19.97968+00:00 | 10s191ms622µs939ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 74 |
| | | seq=6 start=1528141568.365449377,0 epo=3 pro=1528141636.592623635,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.365449377,0 epo=3 | | | | |
| | | pro=1528141636.592623635,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:19.979758+00:00 | 10s191ms701µs740ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=6 | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 74 |
| | | start=1528141568.365449377,0 epo=3 pro=1528141636.592623635,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.365449377,0 epo=3 | | | | |
| | | pro=1528141636.592623635,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:19.979807+00:00 | 10s191ms749µs966ns | read has no clock uncertainty | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | /cockroach.roachpb.Internal/Batch | 72 |
| 2018-06-04 19:47:19.979813+00:00 | 10s191ms756µs67ns | command queue | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | /cockroach.roachpb.Internal/Batch | 72 |
| 2018-06-04 19:47:19.97982+00:00 | 10s191ms763µs418ns | waiting for read lock | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | /cockroach.roachpb.Internal/Batch | 72 |
| 2018-06-04 19:47:19.979836+00:00 | 10s191ms778µs844ns | read completed | [n1,s1,r42/1:/Table/52/1/1{800-900}] | | /cockroach.roachpb.Internal/Batch | 72 |
| 2018-06-04 19:47:19.97992+00:00 | 10s191ms863µs666ns | === SPAN START: dist sender === | | | dist sender | 78 |
| 2018-06-04 19:47:19.979937+00:00 | 10s191ms879µs859ns | querying next range at /Table/52/1/1900 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 78 |
| 2018-06-04 19:47:19.979968+00:00 | 10s191ms911µs555ns | r43: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 78 |
| 2018-06-04 19:47:19.979991+00:00 | 10s191ms934µs729ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 78 |
| 2018-06-04 19:47:19.980006+00:00 | 10s191ms949µs496ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 79 |
| 2018-06-04 19:47:19.98001+00:00 | 10s191ms953µs459ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 79 |
| 2018-06-04 19:47:19.980016+00:00 | 10s191ms958µs989ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 79 |
| 2018-06-04 19:47:19.980125+00:00 | 10s192ms67µs840ns | read-only path | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | /cockroach.roachpb.Internal/Batch | 79 |
| 2018-06-04 19:47:19.980133+00:00 | 10s192ms76µs329ns | request range lease (attempt #1) | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | /cockroach.roachpb.Internal/Batch | 79 |
| 2018-06-04 19:47:19.98014+00:00 | 10s192ms83µs670ns | === SPAN START: request range lease === | | | request range lease | 80 |
| 2018-06-04 19:47:19.980148+00:00 | 10s192ms91µs707ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 81 |
| | | === | | | | |
| 2018-06-04 19:47:19.980184+00:00 | 10s192ms127µs710ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 82 |
| 2018-06-04 19:47:19.980273+00:00 | 10s192ms216µs281ns | === SPAN START: dist sender === | | | dist sender | 83 |
| 2018-06-04 19:47:19.980292+00:00 | 10s192ms235µs678ns | querying next range at /System/NodeLiveness/1 | [s1,r43/1:/Table/52/1/{1900-2000},txn=04a63d23,n1] | | dist sender | 83 |
| 2018-06-04 19:47:19.980323+00:00 | 10s192ms266µs448ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r43/1:/Table/52/1/{1900-2000},txn=04a63d23,n1] | | dist sender | 83 |
| 2018-06-04 19:47:19.980347+00:00 | 10s192ms290µs555ns | sending request to alex-laptop.local:26259 | [s1,r43/1:/Table/52/1/{1900-2000},txn=04a63d23,n1] | | dist sender | 83 |
| 2018-06-04 19:47:19.980379+00:00 | 10s192ms322µs14ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 84 |
| 2018-06-04 19:47:19.980696+00:00 | 10s192ms639µs269ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 412 |
| 2018-06-04 19:47:19.980761+00:00 | 10s192ms704µs622ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 412 |
| 2018-06-04 19:47:19.980773+00:00 | 10s192ms715µs881ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 412 |
| 2018-06-04 19:47:19.980782+00:00 | 10s192ms725µs436ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 412 |
| 2018-06-04 19:47:19.980802+00:00 | 10s192ms744µs865ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 412 |
| 2018-06-04 19:47:19.980838+00:00 | 10s192ms781µs496ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 412 |
| 2018-06-04 19:47:19.980933+00:00 | 10s192ms876µs371ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 412 |
| 2018-06-04 19:47:19.980938+00:00 | 10s192ms880µs885ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 412 |
| 2018-06-04 19:47:19.98163+00:00 | 10s193ms573µs678ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 412 |
| 2018-06-04 19:47:19.981959+00:00 | 10s193ms902µs152ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 412 |
| 2018-06-04 19:47:19.981983+00:00 | 10s193ms926µs740ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 412 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:47:19.982563+00:00 | 10s194ms505µs961ns | heartbeat 1528141649.480192922,0 | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | liveness heartbeat | 82 |
| 2018-06-04 19:47:19.982571+00:00 | 10s194ms514µs394ns | read-write path | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | [async] storage.pendingLeaseRequest: requesting lease | 81 |
| 2018-06-04 19:47:19.982579+00:00 | 10s194ms521µs896ns | applied timestamp cache | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | [async] storage.pendingLeaseRequest: requesting lease | 81 |
| 2018-06-04 19:47:19.982669+00:00 | 10s194ms612µs258ns | evaluated request | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | [async] storage.pendingLeaseRequest: requesting lease | 81 |
| 2018-06-04 19:47:19.982673+00:00 | 10s194ms616µs476ns | acquired {raft,replica}mu | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | [async] storage.pendingLeaseRequest: requesting lease | 81 |
| 2018-06-04 19:47:22.980655+00:00 | 13s192ms598µs98ns | retry proposal 4afe90e9c3732894: reasonTicks | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | [async] storage.pendingLeaseRequest: requesting lease | 81 |
| 2018-06-04 19:47:22.980711+00:00 | 13s192ms654µs179ns | applied timestamp cache | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | [async] storage.pendingLeaseRequest: requesting lease | 81 |
| 2018-06-04 19:47:22.98087+00:00 | 13s192ms813µs742ns | evaluated request | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | [async] storage.pendingLeaseRequest: requesting lease | 81 |
| 2018-06-04 19:47:22.980876+00:00 | 13s192ms818µs968ns | acquired {raft,replica}mu | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | [async] storage.pendingLeaseRequest: requesting lease | 81 |
| 2018-06-04 19:47:24.583476+00:00 | 14s795ms419µs621ns | retry proposal 4de6644536330705: reasonNewLeader | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | [async] storage.pendingLeaseRequest: requesting lease | 81 |
| 2018-06-04 19:47:24.58352+00:00 | 14s795ms462µs879ns | applied timestamp cache | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | [async] storage.pendingLeaseRequest: requesting lease | 81 |
| 2018-06-04 19:47:24.583652+00:00 | 14s795ms595µs48ns | evaluated request | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | [async] storage.pendingLeaseRequest: requesting lease | 81 |
| 2018-06-04 19:47:24.583665+00:00 | 14s795ms608µs684ns | acquired {raft,replica}mu | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | [async] storage.pendingLeaseRequest: requesting lease | 81 |
| 2018-06-04 19:47:24.584328+00:00 | 14s796ms270µs953ns | retry proposal 47cc4a748939a092: reasonNewLeaderOrConfigChange | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | [async] storage.pendingLeaseRequest: requesting lease | 81 |
| 2018-06-04 19:47:24.584363+00:00 | 14s796ms306µs82ns | applied timestamp cache | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | [async] storage.pendingLeaseRequest: requesting lease | 81 |
| 2018-06-04 19:47:24.584441+00:00 | 14s796ms384µs246ns | evaluated request | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | [async] storage.pendingLeaseRequest: requesting lease | 81 |
| 2018-06-04 19:47:24.584452+00:00 | 14s796ms395µs267ns | acquired {raft,replica}mu | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | [async] storage.pendingLeaseRequest: requesting lease | 81 |
| 2018-06-04 19:47:24.58564+00:00 | 14s797ms583µs459ns | command proposed from replica (n1,s1):1 with lease #5 incompatible | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | [async] storage.pendingLeaseRequest: requesting lease | 81 |
| | | to repl=(n1,s1):1 seq=6 start=1528141568.365564620,0 epo=3 | | | | |
| | | pro=1528141639.980135959,0 | | | | |
| 2018-06-04 19:47:24.585658+00:00 | 14s797ms601µs308ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | [async] storage.pendingLeaseRequest: requesting lease | 81 |
| | | seq=6 start=1528141568.365564620,0 epo=3 pro=1528141639.980135959,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.365564620,0 epo=3 | | | | |
| | | pro=1528141639.980135959,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:24.585733+00:00 | 14s797ms676µs506ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=6 | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | [async] storage.pendingLeaseRequest: requesting lease | 81 |
| | | start=1528141568.365564620,0 epo=3 pro=1528141639.980135959,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.365564620,0 epo=3 | | | | |
| | | pro=1528141639.980135959,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:24.585778+00:00 | 14s797ms721µs742ns | read has no clock uncertainty | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | /cockroach.roachpb.Internal/Batch | 79 |
| 2018-06-04 19:47:24.585784+00:00 | 14s797ms727µs217ns | command queue | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | /cockroach.roachpb.Internal/Batch | 79 |
| 2018-06-04 19:47:24.585791+00:00 | 14s797ms734µs170ns | waiting for read lock | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | /cockroach.roachpb.Internal/Batch | 79 |
| 2018-06-04 19:47:24.585804+00:00 | 14s797ms747µs351ns | read completed | [n1,s1,r43/1:/Table/52/1/{1900-2000}] | | /cockroach.roachpb.Internal/Batch | 79 |
| 2018-06-04 19:47:24.585927+00:00 | 14s797ms869µs975ns | === SPAN START: dist sender === | | | dist sender | 85 |
| 2018-06-04 19:47:24.585951+00:00 | 14s797ms894µs111ns | querying next range at /Table/52/1/2000 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 85 |
| 2018-06-04 19:47:24.585982+00:00 | 14s797ms925µs379ns | r44: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 85 |
| 2018-06-04 19:47:24.585998+00:00 | 14s797ms941µs693ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 85 |
| 2018-06-04 19:47:24.586013+00:00 | 14s797ms955µs852ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 86 |
| 2018-06-04 19:47:24.586017+00:00 | 14s797ms960µs11ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 86 |
| 2018-06-04 19:47:24.586022+00:00 | 14s797ms965µs693ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 86 |
| 2018-06-04 19:47:24.586122+00:00 | 14s798ms65µs128ns | read-only path | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | /cockroach.roachpb.Internal/Batch | 86 |
| 2018-06-04 19:47:24.586127+00:00 | 14s798ms70µs334ns | request range lease (attempt #1) | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | /cockroach.roachpb.Internal/Batch | 86 |
| 2018-06-04 19:47:24.586134+00:00 | 14s798ms77µs310ns | === SPAN START: request range lease === | | | request range lease | 87 |
| 2018-06-04 19:47:24.586142+00:00 | 14s798ms85µs802ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 88 |
| | | === | | | | |
| 2018-06-04 19:47:24.586164+00:00 | 14s798ms107µs742ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 89 |
| 2018-06-04 19:47:24.586248+00:00 | 14s798ms191µs281ns | === SPAN START: dist sender === | | | dist sender | 90 |
| 2018-06-04 19:47:24.586261+00:00 | 14s798ms204µs397ns | querying next range at /System/NodeLiveness/1 | [s1,r44/1:/Table/52/1/2{000-100},txn=22722bef,n1] | | dist sender | 90 |
| 2018-06-04 19:47:24.586287+00:00 | 14s798ms230µs99ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r44/1:/Table/52/1/2{000-100},txn=22722bef,n1] | | dist sender | 90 |
| 2018-06-04 19:47:24.586326+00:00 | 14s798ms268µs931ns | sending request to alex-laptop.local:26259 | [s1,r44/1:/Table/52/1/2{000-100},txn=22722bef,n1] | | dist sender | 90 |
| 2018-06-04 19:47:24.586355+00:00 | 14s798ms298µs445ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 91 |
| 2018-06-04 19:47:24.586712+00:00 | 14s798ms654µs957ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 413 |
| 2018-06-04 19:47:24.586803+00:00 | 14s798ms746µs613ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 413 |
| 2018-06-04 19:47:24.586816+00:00 | 14s798ms759µs507ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 413 |
| 2018-06-04 19:47:24.586826+00:00 | 14s798ms769µs346ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 413 |
| 2018-06-04 19:47:24.586842+00:00 | 14s798ms784µs910ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 413 |
| 2018-06-04 19:47:24.586878+00:00 | 14s798ms821µs279ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 413 |
| 2018-06-04 19:47:24.586994+00:00 | 14s798ms936µs820ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 413 |
| 2018-06-04 19:47:24.586998+00:00 | 14s798ms941µs445ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 413 |
| 2018-06-04 19:47:24.58772+00:00 | 14s799ms662µs859ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 413 |
| 2018-06-04 19:47:24.588091+00:00 | 14s800ms34µs25ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 413 |
| 2018-06-04 19:47:24.588117+00:00 | 14s800ms60µs729ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 413 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:47:24.588629+00:00 | 14s800ms571µs870ns | heartbeat 1528141654.086168877,0 | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | liveness heartbeat | 89 |
| 2018-06-04 19:47:24.588639+00:00 | 14s800ms581µs996ns | read-write path | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 88 |
| 2018-06-04 19:47:24.588646+00:00 | 14s800ms589µs431ns | applied timestamp cache | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 88 |
| 2018-06-04 19:47:24.588743+00:00 | 14s800ms686µs429ns | evaluated request | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 88 |
| 2018-06-04 19:47:24.588746+00:00 | 14s800ms689µs677ns | acquired {raft,replica}mu | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 88 |
| 2018-06-04 19:47:27.580386+00:00 | 17s792ms329µs721ns | retry proposal 59345aa9e3705c65: reasonTicks | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 88 |
| 2018-06-04 19:47:27.580448+00:00 | 17s792ms391µs316ns | applied timestamp cache | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 88 |
| 2018-06-04 19:47:27.580613+00:00 | 17s792ms556µs364ns | evaluated request | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 88 |
| 2018-06-04 19:47:27.580618+00:00 | 17s792ms561µs458ns | acquired {raft,replica}mu | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 88 |
| 2018-06-04 19:47:28.584357+00:00 | 18s796ms300µs208ns | retry proposal 39fd5d801bce89f5: reasonNewLeader | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 88 |
| 2018-06-04 19:47:28.584446+00:00 | 18s796ms389µs809ns | applied timestamp cache | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 88 |
| 2018-06-04 19:47:28.584588+00:00 | 18s796ms531µs126ns | evaluated request | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 88 |
| 2018-06-04 19:47:28.584602+00:00 | 18s796ms545µs72ns | acquired {raft,replica}mu | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 88 |
| 2018-06-04 19:47:28.585141+00:00 | 18s797ms84µs556ns | retry proposal 0f6b5ac130473dc9: reasonNewLeaderOrConfigChange | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 88 |
| 2018-06-04 19:47:28.585181+00:00 | 18s797ms124µs773ns | applied timestamp cache | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 88 |
| 2018-06-04 19:47:28.585283+00:00 | 18s797ms226µs300ns | evaluated request | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 88 |
| 2018-06-04 19:47:28.585296+00:00 | 18s797ms239µs246ns | acquired {raft,replica}mu | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 88 |
| 2018-06-04 19:47:28.586535+00:00 | 18s798ms478µs543ns | command proposed from replica (n1,s1):1 with lease #5 incompatible | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 88 |
| | | to repl=(n1,s1):1 seq=6 start=1528141568.365680310,0 epo=3 | | | | |
| | | pro=1528141644.586129448,0 | | | | |
| 2018-06-04 19:47:28.586561+00:00 | 18s798ms504µs668ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 88 |
| | | seq=6 start=1528141568.365680310,0 epo=3 pro=1528141644.586129448,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.365680310,0 epo=3 | | | | |
| | | pro=1528141644.586129448,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:28.586651+00:00 | 18s798ms594µs426ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=6 | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 88 |
| | | start=1528141568.365680310,0 epo=3 pro=1528141644.586129448,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.365680310,0 epo=3 | | | | |
| | | pro=1528141644.586129448,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:28.586703+00:00 | 18s798ms646µs198ns | read has no clock uncertainty | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | /cockroach.roachpb.Internal/Batch | 86 |
| 2018-06-04 19:47:28.586711+00:00 | 18s798ms654µs413ns | command queue | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | /cockroach.roachpb.Internal/Batch | 86 |
| 2018-06-04 19:47:28.586721+00:00 | 18s798ms664µs254ns | waiting for read lock | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | /cockroach.roachpb.Internal/Batch | 86 |
| 2018-06-04 19:47:28.586738+00:00 | 18s798ms681µs225ns | read completed | [n1,s1,r44/1:/Table/52/1/2{000-100}] | | /cockroach.roachpb.Internal/Batch | 86 |
| 2018-06-04 19:47:28.586842+00:00 | 18s798ms785µs165ns | === SPAN START: dist sender === | | | dist sender | 92 |
| 2018-06-04 19:47:28.586859+00:00 | 18s798ms802µs219ns | querying next range at /Table/52/1/2100 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 92 |
| 2018-06-04 19:47:28.58689+00:00 | 18s798ms833µs338ns | r45: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 92 |
| 2018-06-04 19:47:28.586915+00:00 | 18s798ms857µs816ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 92 |
| 2018-06-04 19:47:28.586929+00:00 | 18s798ms872µs32ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 93 |
| 2018-06-04 19:47:28.586933+00:00 | 18s798ms876µs68ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 93 |
| 2018-06-04 19:47:28.586939+00:00 | 18s798ms881µs998ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 93 |
| 2018-06-04 19:47:28.587033+00:00 | 18s798ms976µs556ns | read-only path | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | /cockroach.roachpb.Internal/Batch | 93 |
| 2018-06-04 19:47:28.587039+00:00 | 18s798ms982µs251ns | request range lease (attempt #1) | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | /cockroach.roachpb.Internal/Batch | 93 |
| 2018-06-04 19:47:28.587046+00:00 | 18s798ms989µs694ns | === SPAN START: request range lease === | | | request range lease | 94 |
| 2018-06-04 19:47:28.587054+00:00 | 18s798ms997µs646ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 95 |
| | | === | | | | |
| 2018-06-04 19:47:28.587076+00:00 | 18s799ms19µs6ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 96 |
| 2018-06-04 19:47:28.587154+00:00 | 18s799ms97µs107ns | === SPAN START: dist sender === | | | dist sender | 97 |
| 2018-06-04 19:47:28.587167+00:00 | 18s799ms110µs286ns | querying next range at /System/NodeLiveness/1 | [s1,r45/1:/Table/52/1/2{100-200},txn=40864a8f,n1] | | dist sender | 97 |
| 2018-06-04 19:47:28.587191+00:00 | 18s799ms134µs76ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r45/1:/Table/52/1/2{100-200},txn=40864a8f,n1] | | dist sender | 97 |
| 2018-06-04 19:47:28.587208+00:00 | 18s799ms151µs562ns | sending request to alex-laptop.local:26259 | [s1,r45/1:/Table/52/1/2{100-200},txn=40864a8f,n1] | | dist sender | 97 |
| 2018-06-04 19:47:28.587228+00:00 | 18s799ms171µs558ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 98 |
| 2018-06-04 19:47:28.587538+00:00 | 18s799ms481µs485ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 414 |
| 2018-06-04 19:47:28.587583+00:00 | 18s799ms526µs177ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 414 |
| 2018-06-04 19:47:28.587594+00:00 | 18s799ms537µs80ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 414 |
| 2018-06-04 19:47:28.587603+00:00 | 18s799ms545µs959ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 414 |
| 2018-06-04 19:47:28.587617+00:00 | 18s799ms560µs716ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 414 |
| 2018-06-04 19:47:28.587651+00:00 | 18s799ms594µs68ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 414 |
| 2018-06-04 19:47:28.587753+00:00 | 18s799ms696µs654ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 414 |
| 2018-06-04 19:47:28.587757+00:00 | 18s799ms700µs710ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 414 |
| 2018-06-04 19:47:28.58845+00:00 | 18s800ms393µs292ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 414 |
| 2018-06-04 19:47:28.588778+00:00 | 18s800ms721µs560ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 414 |
| 2018-06-04 19:47:28.588805+00:00 | 18s800ms748µs211ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 414 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:47:28.589405+00:00 | 18s801ms348µs352ns | heartbeat 1528141658.087079500,0 | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | liveness heartbeat | 96 |
| 2018-06-04 19:47:28.589413+00:00 | 18s801ms356µs457ns | read-write path | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 95 |
| 2018-06-04 19:47:28.589423+00:00 | 18s801ms366µs152ns | applied timestamp cache | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 95 |
| 2018-06-04 19:47:28.589508+00:00 | 18s801ms451µs17ns | evaluated request | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 95 |
| 2018-06-04 19:47:28.589511+00:00 | 18s801ms454µs15ns | acquired {raft,replica}mu | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 95 |
| 2018-06-04 19:47:31.575748+00:00 | 21s787ms691µs710ns | retry proposal 2c31750b06a3586f: reasonTicks | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 95 |
| 2018-06-04 19:47:31.57581+00:00 | 21s787ms753µs472ns | applied timestamp cache | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 95 |
| 2018-06-04 19:47:31.57608+00:00 | 21s788ms22µs979ns | evaluated request | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 95 |
| 2018-06-04 19:47:31.576085+00:00 | 21s788ms28µs67ns | acquired {raft,replica}mu | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 95 |
| 2018-06-04 19:47:32.976391+00:00 | 23s188ms333µs972ns | retry proposal 5e38522aef1635f0: reasonNewLeader | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 95 |
| 2018-06-04 19:47:32.976431+00:00 | 23s188ms374µs570ns | applied timestamp cache | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 95 |
| 2018-06-04 19:47:32.976576+00:00 | 23s188ms519µs243ns | evaluated request | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 95 |
| 2018-06-04 19:47:32.976582+00:00 | 23s188ms525µs626ns | acquired {raft,replica}mu | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 95 |
| 2018-06-04 19:47:32.977122+00:00 | 23s189ms64µs936ns | retry proposal 5c3ff670ae7e7ea5: reasonNewLeaderOrConfigChange | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 95 |
| 2018-06-04 19:47:32.977171+00:00 | 23s189ms114µs760ns | applied timestamp cache | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 95 |
| 2018-06-04 19:47:32.97729+00:00 | 23s189ms233µs21ns | evaluated request | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 95 |
| 2018-06-04 19:47:32.977403+00:00 | 23s189ms346µs743ns | acquired {raft,replica}mu | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 95 |
| 2018-06-04 19:47:32.978672+00:00 | 23s190ms615µs139ns | command proposed from replica (n1,s1):1 with lease #5 incompatible | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 95 |
| | | to repl=(n1,s1):1 seq=6 start=1528141568.365795221,0 epo=3 | | | | |
| | | pro=1528141648.587041719,0 | | | | |
| 2018-06-04 19:47:32.97869+00:00 | 23s190ms633µs809ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 95 |
| | | seq=6 start=1528141568.365795221,0 epo=3 pro=1528141648.587041719,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.365795221,0 epo=3 | | | | |
| | | pro=1528141648.587041719,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:32.978769+00:00 | 23s190ms712µs646ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=6 | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 95 |
| | | start=1528141568.365795221,0 epo=3 pro=1528141648.587041719,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.365795221,0 epo=3 | | | | |
| | | pro=1528141648.587041719,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:32.978823+00:00 | 23s190ms766µs139ns | read has no clock uncertainty | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | /cockroach.roachpb.Internal/Batch | 93 |
| 2018-06-04 19:47:32.978828+00:00 | 23s190ms771µs715ns | command queue | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | /cockroach.roachpb.Internal/Batch | 93 |
| 2018-06-04 19:47:32.978835+00:00 | 23s190ms778µs636ns | waiting for read lock | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | /cockroach.roachpb.Internal/Batch | 93 |
| 2018-06-04 19:47:32.97885+00:00 | 23s190ms793µs257ns | read completed | [n1,s1,r45/1:/Table/52/1/2{100-200}] | | /cockroach.roachpb.Internal/Batch | 93 |
| 2018-06-04 19:47:32.978954+00:00 | 23s190ms897µs187ns | === SPAN START: dist sender === | | | dist sender | 99 |
| 2018-06-04 19:47:32.978975+00:00 | 23s190ms918µs36ns | querying next range at /Table/52/1/2200 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 99 |
| 2018-06-04 19:47:32.979019+00:00 | 23s190ms962µs17ns | r46: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 99 |
| 2018-06-04 19:47:32.979041+00:00 | 23s190ms983µs839ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 99 |
| 2018-06-04 19:47:32.979055+00:00 | 23s190ms998µs428ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 100 |
| 2018-06-04 19:47:32.979059+00:00 | 23s191ms2µs468ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 100 |
| 2018-06-04 19:47:32.979065+00:00 | 23s191ms8µs358ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 100 |
| 2018-06-04 19:47:32.979169+00:00 | 23s191ms111µs822ns | read-only path | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | /cockroach.roachpb.Internal/Batch | 100 |
| 2018-06-04 19:47:32.979174+00:00 | 23s191ms117µs477ns | request range lease (attempt #1) | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | /cockroach.roachpb.Internal/Batch | 100 |
| 2018-06-04 19:47:32.979181+00:00 | 23s191ms124µs386ns | === SPAN START: request range lease === | | | request range lease | 101 |
| 2018-06-04 19:47:32.97919+00:00 | 23s191ms133µs593ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| | | === | | | | |
| 2018-06-04 19:47:32.979212+00:00 | 23s191ms155µs313ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 103 |
| 2018-06-04 19:47:32.979292+00:00 | 23s191ms235µs262ns | === SPAN START: dist sender === | | | dist sender | 104 |
| 2018-06-04 19:47:32.979305+00:00 | 23s191ms248µs739ns | querying next range at /System/NodeLiveness/1 | [s1,r46/1:/Table/52/1/2{200-300},txn=52cd37c2,n1] | | dist sender | 104 |
| 2018-06-04 19:47:32.979331+00:00 | 23s191ms274µs366ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r46/1:/Table/52/1/2{200-300},txn=52cd37c2,n1] | | dist sender | 104 |
| 2018-06-04 19:47:32.97935+00:00 | 23s191ms293µs123ns | sending request to alex-laptop.local:26259 | [s1,r46/1:/Table/52/1/2{200-300},txn=52cd37c2,n1] | | dist sender | 104 |
| 2018-06-04 19:47:32.97937+00:00 | 23s191ms313µs745ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 105 |
| 2018-06-04 19:47:32.979691+00:00 | 23s191ms634µs38ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 415 |
| 2018-06-04 19:47:32.979726+00:00 | 23s191ms669µs695ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 415 |
| 2018-06-04 19:47:32.979735+00:00 | 23s191ms678µs53ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 415 |
| 2018-06-04 19:47:32.979741+00:00 | 23s191ms684µs690ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 415 |
| 2018-06-04 19:47:32.979755+00:00 | 23s191ms698µs300ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 415 |
| 2018-06-04 19:47:32.979785+00:00 | 23s191ms727µs831ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 415 |
| 2018-06-04 19:47:32.979856+00:00 | 23s191ms799µs391ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 415 |
| 2018-06-04 19:47:32.97986+00:00 | 23s191ms803µs198ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 415 |
| 2018-06-04 19:47:32.980559+00:00 | 23s192ms501µs874ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 415 |
| 2018-06-04 19:47:32.980949+00:00 | 23s192ms892µs774ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 415 |
| 2018-06-04 19:47:32.980977+00:00 | 23s192ms920µs164ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 415 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:47:32.981566+00:00 | 23s193ms509µs186ns | heartbeat 1528141662.479216401,0 | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | liveness heartbeat | 103 |
| 2018-06-04 19:47:32.981576+00:00 | 23s193ms519µs447ns | read-write path | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:32.981584+00:00 | 23s193ms527µs13ns | applied timestamp cache | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:32.981677+00:00 | 23s193ms620µs309ns | evaluated request | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:32.981682+00:00 | 23s193ms624µs935ns | acquired {raft,replica}mu | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:35.975824+00:00 | 26s187ms767µs339ns | retry proposal 2a70a2d743d86b3b: reasonTicks | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:35.975872+00:00 | 26s187ms815µs811ns | applied timestamp cache | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:35.976013+00:00 | 26s187ms956µs649ns | evaluated request | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:35.976018+00:00 | 26s187ms960µs924ns | acquired {raft,replica}mu | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.381518+00:00 | 26s593ms461µs637ns | retry proposal 6274bf54cc28c241: reasonNewLeader | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.381566+00:00 | 26s593ms509µs205ns | applied timestamp cache | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.381751+00:00 | 26s593ms693µs990ns | evaluated request | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.381756+00:00 | 26s593ms699µs629ns | acquired {raft,replica}mu | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.382375+00:00 | 26s594ms318µs201ns | retry proposal 6826022f38185cfb: reasonNewLeaderOrConfigChange | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.382442+00:00 | 26s594ms385µs32ns | applied timestamp cache | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.382573+00:00 | 26s594ms516µs173ns | evaluated request | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.382672+00:00 | 26s594ms615µs415ns | acquired {raft,replica}mu | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.383609+00:00 | 26s595ms552µs158ns | retry proposal 5cd3cfd400ba12b5: reasonNewLeader | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.383624+00:00 | 26s595ms566µs979ns | applied timestamp cache | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.383684+00:00 | 26s595ms626µs926ns | evaluated request | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.383687+00:00 | 26s595ms630µs465ns | acquired {raft,replica}mu | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.384347+00:00 | 26s596ms290µs655ns | retry proposal 2b1b123aceb70e96: reasonNewLeader | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.384405+00:00 | 26s596ms348µs601ns | applied timestamp cache | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.384497+00:00 | 26s596ms440µs353ns | evaluated request | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.384511+00:00 | 26s596ms453µs903ns | acquired {raft,replica}mu | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.384982+00:00 | 26s596ms925µs545ns | retry proposal 385688585d2dcc7b: reasonNewLeaderOrConfigChange | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.385014+00:00 | 26s596ms957µs95ns | applied timestamp cache | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.385081+00:00 | 26s597ms24µs309ns | evaluated request | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.385091+00:00 | 26s597ms34µs739ns | acquired {raft,replica}mu | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| 2018-06-04 19:47:36.385878+00:00 | 26s597ms821µs359ns | command proposed from replica (n1,s1):1 with lease #5 incompatible | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| | | to repl=(n1,s1):1 seq=6 start=1528141568.365908415,0 epo=3 | | | | |
| | | pro=1528141652.979176881,0 | | | | |
| 2018-06-04 19:47:36.385901+00:00 | 26s597ms844µs368ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| | | seq=6 start=1528141568.365908415,0 epo=3 pro=1528141652.979176881,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.365908415,0 epo=3 | | | | |
| | | pro=1528141652.979176881,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:36.386039+00:00 | 26s597ms982µs699ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=6 | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 102 |
| | | start=1528141568.365908415,0 epo=3 pro=1528141652.979176881,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.365908415,0 epo=3 | | | | |
| | | pro=1528141652.979176881,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:36.386162+00:00 | 26s598ms104µs816ns | read has no clock uncertainty | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | /cockroach.roachpb.Internal/Batch | 100 |
| 2018-06-04 19:47:36.386167+00:00 | 26s598ms110µs383ns | command queue | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | /cockroach.roachpb.Internal/Batch | 100 |
| 2018-06-04 19:47:36.386175+00:00 | 26s598ms118µs580ns | waiting for read lock | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | /cockroach.roachpb.Internal/Batch | 100 |
| 2018-06-04 19:47:36.386193+00:00 | 26s598ms135µs912ns | read completed | [n1,s1,r46/1:/Table/52/1/2{200-300}] | | /cockroach.roachpb.Internal/Batch | 100 |
| 2018-06-04 19:47:36.38629+00:00 | 26s598ms233µs105ns | === SPAN START: dist sender === | | | dist sender | 106 |
| 2018-06-04 19:47:36.386305+00:00 | 26s598ms248µs700ns | querying next range at /Table/52/1/2300 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 106 |
| 2018-06-04 19:47:36.386338+00:00 | 26s598ms281µs148ns | r47: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 106 |
| 2018-06-04 19:47:36.386361+00:00 | 26s598ms304µs552ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 106 |
| 2018-06-04 19:47:36.386395+00:00 | 26s598ms337µs950ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 107 |
| 2018-06-04 19:47:36.386398+00:00 | 26s598ms341µs759ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 107 |
| 2018-06-04 19:47:36.386404+00:00 | 26s598ms346µs901ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 107 |
| 2018-06-04 19:47:36.386501+00:00 | 26s598ms444µs85ns | read-only path | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | /cockroach.roachpb.Internal/Batch | 107 |
| 2018-06-04 19:47:36.386506+00:00 | 26s598ms449µs74ns | request range lease (attempt #1) | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | /cockroach.roachpb.Internal/Batch | 107 |
| 2018-06-04 19:47:36.386512+00:00 | 26s598ms455µs473ns | === SPAN START: request range lease === | | | request range lease | 108 |
| 2018-06-04 19:47:36.386519+00:00 | 26s598ms462µs279ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 109 |
| | | === | | | | |
| 2018-06-04 19:47:36.386539+00:00 | 26s598ms482µs399ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 110 |
| 2018-06-04 19:47:36.386607+00:00 | 26s598ms550µs722ns | === SPAN START: dist sender === | | | dist sender | 111 |
| 2018-06-04 19:47:36.386621+00:00 | 26s598ms564µs355ns | querying next range at /System/NodeLiveness/1 | [s1,r47/1:/Table/52/1/2{300-400},txn=08707ce3,n1] | | dist sender | 111 |
| 2018-06-04 19:47:36.386645+00:00 | 26s598ms588µs736ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r47/1:/Table/52/1/2{300-400},txn=08707ce3,n1] | | dist sender | 111 |
| 2018-06-04 19:47:36.386662+00:00 | 26s598ms605µs665ns | sending request to alex-laptop.local:26259 | [s1,r47/1:/Table/52/1/2{300-400},txn=08707ce3,n1] | | dist sender | 111 |
| 2018-06-04 19:47:36.386681+00:00 | 26s598ms624µs279ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 112 |
| 2018-06-04 19:47:36.386971+00:00 | 26s598ms914µs595ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 416 |
| 2018-06-04 19:47:36.387016+00:00 | 26s598ms959µs187ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 416 |
| 2018-06-04 19:47:36.387024+00:00 | 26s598ms966µs848ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 416 |
| 2018-06-04 19:47:36.38703+00:00 | 26s598ms973µs764ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 416 |
| 2018-06-04 19:47:36.387043+00:00 | 26s598ms986µs214ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 416 |
| 2018-06-04 19:47:36.387085+00:00 | 26s599ms28µs721ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 416 |
| 2018-06-04 19:47:36.387154+00:00 | 26s599ms97µs583ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 416 |
| 2018-06-04 19:47:36.387158+00:00 | 26s599ms100µs915ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 416 |
| 2018-06-04 19:47:36.387807+00:00 | 26s599ms750µs202ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 416 |
| 2018-06-04 19:47:36.388111+00:00 | 26s600ms54µs237ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 416 |
| 2018-06-04 19:47:36.388133+00:00 | 26s600ms76µs49ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 416 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:47:36.388564+00:00 | 26s600ms507µs624ns | heartbeat 1528141665.886542838,0 | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | liveness heartbeat | 110 |
| 2018-06-04 19:47:36.388572+00:00 | 26s600ms514µs871ns | read-write path | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 109 |
| 2018-06-04 19:47:36.388579+00:00 | 26s600ms522µs724ns | applied timestamp cache | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 109 |
| 2018-06-04 19:47:36.388659+00:00 | 26s600ms601µs979ns | evaluated request | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 109 |
| 2018-06-04 19:47:36.388661+00:00 | 26s600ms604µs683ns | acquired {raft,replica}mu | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 109 |
| 2018-06-04 19:47:39.377164+00:00 | 29s589ms107µs209ns | retry proposal 3bcaa974f751a22a: reasonTicks | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 109 |
| 2018-06-04 19:47:39.377221+00:00 | 29s589ms163µs823ns | applied timestamp cache | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 109 |
| 2018-06-04 19:47:39.377431+00:00 | 29s589ms374µs402ns | evaluated request | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 109 |
| 2018-06-04 19:47:39.377437+00:00 | 29s589ms379µs842ns | acquired {raft,replica}mu | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 109 |
| 2018-06-04 19:47:39.975672+00:00 | 30s187ms615µs649ns | retry proposal 350a342b814a416f: reasonNewLeader | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 109 |
| 2018-06-04 19:47:39.975711+00:00 | 30s187ms653µs995ns | applied timestamp cache | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 109 |
| 2018-06-04 19:47:39.975827+00:00 | 30s187ms770µs358ns | evaluated request | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 109 |
| 2018-06-04 19:47:39.975832+00:00 | 30s187ms775µs218ns | acquired {raft,replica}mu | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 109 |
| 2018-06-04 19:47:39.976189+00:00 | 30s188ms131µs908ns | retry proposal 63a110c32453933f: reasonNewLeaderOrConfigChange | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 109 |
| 2018-06-04 19:47:39.976215+00:00 | 30s188ms157µs990ns | applied timestamp cache | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 109 |
| 2018-06-04 19:47:39.976305+00:00 | 30s188ms247µs924ns | evaluated request | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 109 |
| 2018-06-04 19:47:39.976308+00:00 | 30s188ms251µs790ns | acquired {raft,replica}mu | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 109 |
| 2018-06-04 19:47:39.977483+00:00 | 30s189ms426µs586ns | command proposed from replica (n1,s1):1 with lease #3 incompatible | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 109 |
| | | to repl=(n1,s1):1 seq=4 start=1528141568.366026368,0 epo=3 | | | | |
| | | pro=1528141656.386508181,0 | | | | |
| 2018-06-04 19:47:39.9775+00:00 | 30s189ms442µs948ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 109 |
| | | seq=4 start=1528141568.366026368,0 epo=3 pro=1528141656.386508181,0 | | | | |
| | | with repl=(n1,s1):1 seq=4 start=1528141568.366026368,0 epo=3 | | | | |
| | | pro=1528141656.386508181,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:39.97756+00:00 | 30s189ms502µs953ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=4 | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 109 |
| | | start=1528141568.366026368,0 epo=3 pro=1528141656.386508181,0 | | | | |
| | | with repl=(n1,s1):1 seq=4 start=1528141568.366026368,0 epo=3 | | | | |
| | | pro=1528141656.386508181,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:39.977603+00:00 | 30s189ms545µs913ns | read has no clock uncertainty | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | /cockroach.roachpb.Internal/Batch | 107 |
| 2018-06-04 19:47:39.977608+00:00 | 30s189ms551µs34ns | command queue | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | /cockroach.roachpb.Internal/Batch | 107 |
| 2018-06-04 19:47:39.977614+00:00 | 30s189ms557µs196ns | waiting for read lock | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | /cockroach.roachpb.Internal/Batch | 107 |
| 2018-06-04 19:47:39.977625+00:00 | 30s189ms568µs76ns | read completed | [n1,s1,r47/1:/Table/52/1/2{300-400}] | | /cockroach.roachpb.Internal/Batch | 107 |
| 2018-06-04 19:47:39.977706+00:00 | 30s189ms649µs241ns | === SPAN START: dist sender === | | | dist sender | 113 |
| 2018-06-04 19:47:39.97772+00:00 | 30s189ms663µs317ns | querying next range at /Table/52/1/2400 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 113 |
| 2018-06-04 19:47:39.97775+00:00 | 30s189ms693µs772ns | r48: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 113 |
| 2018-06-04 19:47:39.977774+00:00 | 30s189ms717µs325ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 113 |
| 2018-06-04 19:47:39.977788+00:00 | 30s189ms730µs912ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 114 |
| 2018-06-04 19:47:39.977792+00:00 | 30s189ms734µs968ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 114 |
| 2018-06-04 19:47:39.977797+00:00 | 30s189ms739µs957ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 114 |
| 2018-06-04 19:47:39.977886+00:00 | 30s189ms828µs913ns | read-only path | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | /cockroach.roachpb.Internal/Batch | 114 |
| 2018-06-04 19:47:39.977891+00:00 | 30s189ms834µs206ns | request range lease (attempt #1) | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | /cockroach.roachpb.Internal/Batch | 114 |
| 2018-06-04 19:47:39.977897+00:00 | 30s189ms840µs417ns | === SPAN START: request range lease === | | | request range lease | 115 |
| 2018-06-04 19:47:39.977904+00:00 | 30s189ms847µs813ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| | | === | | | | |
| 2018-06-04 19:47:39.977925+00:00 | 30s189ms868µs348ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 117 |
| 2018-06-04 19:47:39.978001+00:00 | 30s189ms943µs958ns | === SPAN START: dist sender === | | | dist sender | 118 |
| 2018-06-04 19:47:39.978013+00:00 | 30s189ms955µs866ns | querying next range at /System/NodeLiveness/1 | [s1,r48/1:/Table/52/1/2{400-500},txn=8994b536,n1] | | dist sender | 118 |
| 2018-06-04 19:47:39.978037+00:00 | 30s189ms979µs991ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r48/1:/Table/52/1/2{400-500},txn=8994b536,n1] | | dist sender | 118 |
| 2018-06-04 19:47:39.978054+00:00 | 30s189ms996µs954ns | sending request to alex-laptop.local:26259 | [s1,r48/1:/Table/52/1/2{400-500},txn=8994b536,n1] | | dist sender | 118 |
| 2018-06-04 19:47:39.978073+00:00 | 30s190ms16µs722ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 119 |
| 2018-06-04 19:47:39.978383+00:00 | 30s190ms326µs714ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 417 |
| 2018-06-04 19:47:39.978433+00:00 | 30s190ms376µs651ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 417 |
| 2018-06-04 19:47:39.978441+00:00 | 30s190ms384µs622ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 417 |
| 2018-06-04 19:47:39.978448+00:00 | 30s190ms391µs52ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 417 |
| 2018-06-04 19:47:39.978461+00:00 | 30s190ms404µs513ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 417 |
| 2018-06-04 19:47:39.978503+00:00 | 30s190ms446µs8ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 417 |
| 2018-06-04 19:47:39.978584+00:00 | 30s190ms527µs234ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 417 |
| 2018-06-04 19:47:39.978588+00:00 | 30s190ms531µs335ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 417 |
| 2018-06-04 19:47:39.979278+00:00 | 30s191ms221µs22ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 417 |
| 2018-06-04 19:47:39.979547+00:00 | 30s191ms490µs225ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 417 |
| 2018-06-04 19:47:39.97957+00:00 | 30s191ms513µs687ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 417 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:47:39.980001+00:00 | 30s191ms944µs431ns | heartbeat 1528141669.477928955,0 | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | liveness heartbeat | 117 |
| 2018-06-04 19:47:39.98001+00:00 | 30s191ms953µs178ns | read-write path | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:39.980019+00:00 | 30s191ms962µs643ns | applied timestamp cache | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:39.980097+00:00 | 30s192ms40µs709ns | evaluated request | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:39.9801+00:00 | 30s192ms43µs612ns | acquired {raft,replica}mu | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:42.978613+00:00 | 33s190ms556µs588ns | retry proposal 2f70fc7d82040ec0: reasonTicks | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:42.978722+00:00 | 33s190ms665µs538ns | applied timestamp cache | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:42.978905+00:00 | 33s190ms848µs610ns | evaluated request | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:42.97891+00:00 | 33s190ms853µs655ns | acquired {raft,replica}mu | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.780041+00:00 | 34s991ms984µs557ns | retry proposal 4a6d3173b0e969bb: reasonNewLeader | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.780077+00:00 | 34s992ms20µs814ns | applied timestamp cache | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.780214+00:00 | 34s992ms157µs671ns | evaluated request | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.78022+00:00 | 34s992ms163µs286ns | acquired {raft,replica}mu | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.780841+00:00 | 34s992ms784µs276ns | retry proposal 1a6ed69599d54b71: reasonNewLeaderOrConfigChange | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.780905+00:00 | 34s992ms848µs184ns | applied timestamp cache | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.781028+00:00 | 34s992ms970µs909ns | evaluated request | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.781144+00:00 | 34s993ms87µs565ns | acquired {raft,replica}mu | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.782201+00:00 | 34s994ms143µs999ns | retry proposal 062f66ad9c729644: reasonNewLeader | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.782217+00:00 | 34s994ms160µs793ns | applied timestamp cache | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.782278+00:00 | 34s994ms221µs676ns | evaluated request | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.782281+00:00 | 34s994ms224µs99ns | acquired {raft,replica}mu | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.782923+00:00 | 34s994ms866µs403ns | retry proposal 4a35391fc9318a81: reasonNewLeader | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.783008+00:00 | 34s994ms950µs924ns | applied timestamp cache | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.783124+00:00 | 34s995ms67µs226ns | evaluated request | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.783137+00:00 | 34s995ms80µs499ns | acquired {raft,replica}mu | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.783608+00:00 | 34s995ms551µs15ns | retry proposal 50cab8e60f41670c: reasonNewLeaderOrConfigChange | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.783633+00:00 | 34s995ms576µs498ns | applied timestamp cache | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.78369+00:00 | 34s995ms633µs805ns | evaluated request | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.783699+00:00 | 34s995ms641µs817ns | acquired {raft,replica}mu | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| 2018-06-04 19:47:44.78447+00:00 | 34s996ms412µs943ns | command proposed from replica (n1,s1):1 with lease #3 incompatible | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| | | to repl=(n1,s1):1 seq=4 start=1528141568.366146142,0 epo=3 | | | | |
| | | pro=1528141659.977893295,0 | | | | |
| 2018-06-04 19:47:44.784492+00:00 | 34s996ms434µs827ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| | | seq=4 start=1528141568.366146142,0 epo=3 pro=1528141659.977893295,0 | | | | |
| | | with repl=(n1,s1):1 seq=4 start=1528141568.366146142,0 epo=3 | | | | |
| | | pro=1528141659.977893295,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:44.784588+00:00 | 34s996ms531µs335ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=4 | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 116 |
| | | start=1528141568.366146142,0 epo=3 pro=1528141659.977893295,0 | | | | |
| | | with repl=(n1,s1):1 seq=4 start=1528141568.366146142,0 epo=3 | | | | |
| | | pro=1528141659.977893295,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:44.784646+00:00 | 34s996ms589µs480ns | read has no clock uncertainty | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | /cockroach.roachpb.Internal/Batch | 114 |
| 2018-06-04 19:47:44.784654+00:00 | 34s996ms596µs923ns | command queue | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | /cockroach.roachpb.Internal/Batch | 114 |
| 2018-06-04 19:47:44.784663+00:00 | 34s996ms606µs471ns | waiting for read lock | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | /cockroach.roachpb.Internal/Batch | 114 |
| 2018-06-04 19:47:44.78468+00:00 | 34s996ms623µs170ns | read completed | [n1,s1,r48/1:/Table/52/1/2{400-500}] | | /cockroach.roachpb.Internal/Batch | 114 |
| 2018-06-04 19:47:44.784778+00:00 | 34s996ms721µs336ns | === SPAN START: dist sender === | | | dist sender | 120 |
| 2018-06-04 19:47:44.784796+00:00 | 34s996ms739µs364ns | querying next range at /Table/52/1/2500 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 120 |
| 2018-06-04 19:47:44.784827+00:00 | 34s996ms769µs857ns | r49: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 120 |
| 2018-06-04 19:47:44.784849+00:00 | 34s996ms792µs246ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 120 |
| 2018-06-04 19:47:44.784864+00:00 | 34s996ms806µs993ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 121 |
| 2018-06-04 19:47:44.784868+00:00 | 34s996ms811µs41ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 121 |
| 2018-06-04 19:47:44.784876+00:00 | 34s996ms819µs39ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 121 |
| 2018-06-04 19:47:44.784974+00:00 | 34s996ms916µs858ns | read-only path | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | /cockroach.roachpb.Internal/Batch | 121 |
| 2018-06-04 19:47:44.784979+00:00 | 34s996ms922µs144ns | request range lease (attempt #1) | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | /cockroach.roachpb.Internal/Batch | 121 |
| 2018-06-04 19:47:44.784989+00:00 | 34s996ms932µs27ns | === SPAN START: request range lease === | | | request range lease | 122 |
| 2018-06-04 19:47:44.784996+00:00 | 34s996ms939µs220ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 123 |
| | | === | | | | |
| 2018-06-04 19:47:44.785017+00:00 | 34s996ms960µs186ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 124 |
| 2018-06-04 19:47:44.785107+00:00 | 34s997ms49µs910ns | === SPAN START: dist sender === | | | dist sender | 125 |
| 2018-06-04 19:47:44.785119+00:00 | 34s997ms62µs447ns | querying next range at /System/NodeLiveness/1 | [s1,r49/1:/Table/52/1/2{500-600},txn=9682a97e,n1] | | dist sender | 125 |
| 2018-06-04 19:47:44.785141+00:00 | 34s997ms84µs335ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r49/1:/Table/52/1/2{500-600},txn=9682a97e,n1] | | dist sender | 125 |
| 2018-06-04 19:47:44.785157+00:00 | 34s997ms100µs112ns | sending request to alex-laptop.local:26259 | [s1,r49/1:/Table/52/1/2{500-600},txn=9682a97e,n1] | | dist sender | 125 |
| 2018-06-04 19:47:44.785175+00:00 | 34s997ms118µs762ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 126 |
| 2018-06-04 19:47:44.785459+00:00 | 34s997ms402µs13ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 418 |
| 2018-06-04 19:47:44.785501+00:00 | 34s997ms444µs667ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 418 |
| 2018-06-04 19:47:44.785509+00:00 | 34s997ms452µs218ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 418 |
| 2018-06-04 19:47:44.785516+00:00 | 34s997ms458µs946ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 418 |
| 2018-06-04 19:47:44.785528+00:00 | 34s997ms471µs509ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 418 |
| 2018-06-04 19:47:44.785556+00:00 | 34s997ms499µs335ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 418 |
| 2018-06-04 19:47:44.785634+00:00 | 34s997ms577µs747ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 418 |
| 2018-06-04 19:47:44.785637+00:00 | 34s997ms580µs624ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 418 |
| 2018-06-04 19:47:44.786174+00:00 | 34s998ms116µs930ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 418 |
| 2018-06-04 19:47:44.786426+00:00 | 34s998ms368µs858ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 418 |
| 2018-06-04 19:47:44.786448+00:00 | 34s998ms391µs108ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 418 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:47:44.786902+00:00 | 34s998ms845µs731ns | heartbeat 1528141674.285020605,0 | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | liveness heartbeat | 124 |
| 2018-06-04 19:47:44.786914+00:00 | 34s998ms857µs181ns | read-write path | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 123 |
| 2018-06-04 19:47:44.786932+00:00 | 34s998ms875µs562ns | applied timestamp cache | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 123 |
| 2018-06-04 19:47:44.787033+00:00 | 34s998ms975µs888ns | evaluated request | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 123 |
| 2018-06-04 19:47:44.787036+00:00 | 34s998ms979µs288ns | acquired {raft,replica}mu | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 123 |
| 2018-06-04 19:47:47.775694+00:00 | 37s987ms637µs129ns | retry proposal 5c26f84bfa7d4b0a: reasonTicks | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 123 |
| 2018-06-04 19:47:47.77578+00:00 | 37s987ms723µs365ns | applied timestamp cache | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 123 |
| 2018-06-04 19:47:47.775936+00:00 | 37s987ms878µs945ns | evaluated request | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 123 |
| 2018-06-04 19:47:47.77594+00:00 | 37s987ms883µs649ns | acquired {raft,replica}mu | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 123 |
| 2018-06-04 19:47:48.181303+00:00 | 38s393ms246µs631ns | retry proposal 6313d5e8358a1308: reasonNewLeader | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 123 |
| 2018-06-04 19:47:48.181441+00:00 | 38s393ms384µs651ns | applied timestamp cache | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 123 |
| 2018-06-04 19:47:48.181607+00:00 | 38s393ms550µs703ns | evaluated request | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 123 |
| 2018-06-04 19:47:48.181626+00:00 | 38s393ms569µs740ns | acquired {raft,replica}mu | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 123 |
| 2018-06-04 19:47:48.182153+00:00 | 38s394ms96µs173ns | retry proposal 28538949b250a0d0: reasonNewLeaderOrConfigChange | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 123 |
| 2018-06-04 19:47:48.18223+00:00 | 38s394ms173µs89ns | applied timestamp cache | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 123 |
| 2018-06-04 19:47:48.182365+00:00 | 38s394ms307µs919ns | evaluated request | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 123 |
| 2018-06-04 19:47:48.182427+00:00 | 38s394ms370µs668ns | acquired {raft,replica}mu | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 123 |
| 2018-06-04 19:47:48.183579+00:00 | 38s395ms522µs713ns | command proposed from replica (n1,s1):1 with lease #5 incompatible | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 123 |
| | | to repl=(n1,s1):1 seq=6 start=1528141568.366263716,0 epo=3 | | | | |
| | | pro=1528141664.784981501,0 | | | | |
| 2018-06-04 19:47:48.183613+00:00 | 38s395ms555µs856ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 123 |
| | | seq=6 start=1528141568.366263716,0 epo=3 pro=1528141664.784981501,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.366263716,0 epo=3 | | | | |
| | | pro=1528141664.784981501,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:48.183715+00:00 | 38s395ms658µs637ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=6 | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 123 |
| | | start=1528141568.366263716,0 epo=3 pro=1528141664.784981501,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.366263716,0 epo=3 | | | | |
| | | pro=1528141664.784981501,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:48.183781+00:00 | 38s395ms724µs52ns | read has no clock uncertainty | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | /cockroach.roachpb.Internal/Batch | 121 |
| 2018-06-04 19:47:48.183789+00:00 | 38s395ms732µs685ns | command queue | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | /cockroach.roachpb.Internal/Batch | 121 |
| 2018-06-04 19:47:48.183799+00:00 | 38s395ms742µs697ns | waiting for read lock | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | /cockroach.roachpb.Internal/Batch | 121 |
| 2018-06-04 19:47:48.183819+00:00 | 38s395ms762µs575ns | read completed | [n1,s1,r49/1:/Table/52/1/2{500-600}] | | /cockroach.roachpb.Internal/Batch | 121 |
| 2018-06-04 19:47:48.18394+00:00 | 38s395ms883µs566ns | === SPAN START: dist sender === | | | dist sender | 127 |
| 2018-06-04 19:47:48.183966+00:00 | 38s395ms909µs529ns | querying next range at /Table/52/1/2600 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 127 |
| 2018-06-04 19:47:48.184005+00:00 | 38s395ms948µs85ns | === SPAN START: range lookup === | | | range lookup | 128 |
| 2018-06-04 19:47:48.184043+00:00 | 38s395ms986µs249ns | === SPAN START: dist sender === | | | dist sender | 129 |
| 2018-06-04 19:47:48.184092+00:00 | 38s396ms35µs699ns | querying next range at /Meta2/Table/52/1/2600/NULL | [client=[::1]:57501,user=root,txn=9352587e,range-lookup=/Table/52/1/2600,n1] | | dist sender | 129 |
| 2018-06-04 19:47:48.184168+00:00 | 38s396ms111µs431ns | r1: sending batch 1 Scan to (n2,s2):3 | [client=[::1]:57501,user=root,txn=9352587e,range-lookup=/Table/52/1/2600,n1] | | dist sender | 129 |
| 2018-06-04 19:47:48.184226+00:00 | 38s396ms168µs991ns | sending request to alex-laptop.local:26259 | [client=[::1]:57501,user=root,txn=9352587e,range-lookup=/Table/52/1/2600,n1] | | dist sender | 129 |
| 2018-06-04 19:47:48.18424+00:00 | 38s396ms183µs347ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 130 |
| 2018-06-04 19:47:48.185344+00:00 | 38s397ms287µs362ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 419 |
| 2018-06-04 19:47:48.185402+00:00 | 38s397ms344µs973ns | 1 Scan | [n2] | | /cockroach.roachpb.Internal/Batch | 419 |
| 2018-06-04 19:47:48.185414+00:00 | 38s397ms356µs985ns | executing 1 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 419 |
| 2018-06-04 19:47:48.185426+00:00 | 38s397ms369µs587ns | read-only path | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 419 |
| 2018-06-04 19:47:48.185451+00:00 | 38s397ms394µs564ns | command queue | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 419 |
| 2018-06-04 19:47:48.185455+00:00 | 38s397ms398µs334ns | operation accepts inconsistent results | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 419 |
| 2018-06-04 19:47:48.18546+00:00 | 38s397ms403µs66ns | waiting for read lock | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 419 |
| 2018-06-04 19:47:48.185941+00:00 | 38s397ms884µs722ns | engine stats: {InternalDeleteSkippedCount:10 TimeBoundNumSSTs:0} | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 419 |
| 2018-06-04 19:47:48.185958+00:00 | 38s397ms900µs905ns | read completed | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 419 |
| 2018-06-04 19:47:48.186809+00:00 | 38s398ms752µs436ns | looked up range descriptor | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 127 |
| 2018-06-04 19:47:48.186847+00:00 | 38s398ms790µs266ns | r50: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 127 |
| 2018-06-04 19:47:48.18687+00:00 | 38s398ms813µs266ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 127 |
| 2018-06-04 19:47:48.186911+00:00 | 38s398ms854µs107ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 131 |
| 2018-06-04 19:47:48.186919+00:00 | 38s398ms861µs884ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 131 |
| 2018-06-04 19:47:48.186927+00:00 | 38s398ms870µs67ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 131 |
| 2018-06-04 19:47:48.187093+00:00 | 38s399ms36µs184ns | read-only path | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | /cockroach.roachpb.Internal/Batch | 131 |
| 2018-06-04 19:47:48.187099+00:00 | 38s399ms42µs340ns | request range lease (attempt #1) | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | /cockroach.roachpb.Internal/Batch | 131 |
| 2018-06-04 19:47:48.187109+00:00 | 38s399ms52µs8ns | === SPAN START: request range lease === | | | request range lease | 132 |
| 2018-06-04 19:47:48.187119+00:00 | 38s399ms62µs556ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| | | === | | | | |
| 2018-06-04 19:47:48.187146+00:00 | 38s399ms89µs326ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 134 |
| 2018-06-04 19:47:48.187247+00:00 | 38s399ms190µs526ns | === SPAN START: dist sender === | | | dist sender | 135 |
| 2018-06-04 19:47:48.187269+00:00 | 38s399ms212µs536ns | querying next range at /System/NodeLiveness/1 | [s1,r50/1:/Table/52/1/2{600-700},txn=636dd461,n1] | | dist sender | 135 |
| 2018-06-04 19:47:48.1873+00:00 | 38s399ms243µs467ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r50/1:/Table/52/1/2{600-700},txn=636dd461,n1] | | dist sender | 135 |
| 2018-06-04 19:47:48.187324+00:00 | 38s399ms267µs528ns | sending request to alex-laptop.local:26259 | [s1,r50/1:/Table/52/1/2{600-700},txn=636dd461,n1] | | dist sender | 135 |
| 2018-06-04 19:47:48.187349+00:00 | 38s399ms292µs578ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 136 |
| 2018-06-04 19:47:48.187774+00:00 | 38s399ms716µs888ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 420 |
| 2018-06-04 19:47:48.187812+00:00 | 38s399ms754µs959ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 420 |
| 2018-06-04 19:47:48.187822+00:00 | 38s399ms765µs483ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 420 |
| 2018-06-04 19:47:48.187831+00:00 | 38s399ms773µs981ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 420 |
| 2018-06-04 19:47:48.187845+00:00 | 38s399ms788µs146ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 420 |
| 2018-06-04 19:47:48.187876+00:00 | 38s399ms819µs600ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 420 |
| 2018-06-04 19:47:48.187955+00:00 | 38s399ms898µs780ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 420 |
| 2018-06-04 19:47:48.18796+00:00 | 38s399ms903µs588ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 420 |
| 2018-06-04 19:47:48.188925+00:00 | 38s400ms868µs521ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 420 |
| 2018-06-04 19:47:48.189275+00:00 | 38s401ms217µs896ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 420 |
| 2018-06-04 19:47:48.189298+00:00 | 38s401ms241µs578ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 420 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:47:48.189809+00:00 | 38s401ms751µs942ns | heartbeat 1528141677.687150618,0 | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | liveness heartbeat | 134 |
| 2018-06-04 19:47:48.189819+00:00 | 38s401ms762µs264ns | read-write path | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:48.189828+00:00 | 38s401ms771µs584ns | applied timestamp cache | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:48.189929+00:00 | 38s401ms872µs240ns | evaluated request | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:48.189932+00:00 | 38s401ms875µs629ns | acquired {raft,replica}mu | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:51.179709+00:00 | 41s391ms652µs241ns | retry proposal 7d54c8db71faac10: reasonTicks | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:51.179767+00:00 | 41s391ms710µs471ns | applied timestamp cache | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:51.179945+00:00 | 41s391ms888µs14ns | evaluated request | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:51.17995+00:00 | 41s391ms892µs925ns | acquired {raft,replica}mu | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.575863+00:00 | 42s787ms806µs1ns | retry proposal 6741ab4b56444db1: reasonNewLeader | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.575912+00:00 | 42s787ms855µs339ns | applied timestamp cache | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.576061+00:00 | 42s788ms4µs256ns | evaluated request | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.576066+00:00 | 42s788ms9µs272ns | acquired {raft,replica}mu | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.576729+00:00 | 42s788ms672µs172ns | retry proposal 7314d6ab358ee114: reasonNewLeaderOrConfigChange | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.576799+00:00 | 42s788ms742µs358ns | applied timestamp cache | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.577005+00:00 | 42s788ms948µs176ns | evaluated request | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.57711+00:00 | 42s789ms53µs710ns | acquired {raft,replica}mu | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.578187+00:00 | 42s790ms130µs205ns | retry proposal 61a9761caf5513fc: reasonNewLeader | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.578202+00:00 | 42s790ms145µs227ns | applied timestamp cache | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.57829+00:00 | 42s790ms233µs230ns | evaluated request | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.578293+00:00 | 42s790ms236µs810ns | acquired {raft,replica}mu | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.579026+00:00 | 42s790ms969µs274ns | retry proposal 44b8f07bea2f316d: reasonNewLeader | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.579099+00:00 | 42s791ms42µs203ns | applied timestamp cache | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.579183+00:00 | 42s791ms126µs273ns | evaluated request | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.579194+00:00 | 42s791ms137µs747ns | acquired {raft,replica}mu | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.579654+00:00 | 42s791ms597µs412ns | retry proposal 5fafeaecad95f3e1: reasonNewLeaderOrConfigChange | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.579686+00:00 | 42s791ms628µs933ns | applied timestamp cache | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.579741+00:00 | 42s791ms684µs206ns | evaluated request | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.579748+00:00 | 42s791ms691µs236ns | acquired {raft,replica}mu | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| 2018-06-04 19:47:52.580487+00:00 | 42s792ms430µs617ns | command proposed from replica (n1,s1):1 with lease #5 incompatible | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| | | to repl=(n1,s1):1 seq=6 start=1528141568.366379826,0 epo=3 | | | | |
| | | pro=1528141668.187101719,0 | | | | |
| 2018-06-04 19:47:52.580511+00:00 | 42s792ms454µs750ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| | | seq=6 start=1528141568.366379826,0 epo=3 pro=1528141668.187101719,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.366379826,0 epo=3 | | | | |
| | | pro=1528141668.187101719,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:52.580615+00:00 | 42s792ms558µs374ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=6 | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 133 |
| | | start=1528141568.366379826,0 epo=3 pro=1528141668.187101719,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.366379826,0 epo=3 | | | | |
| | | pro=1528141668.187101719,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:52.580667+00:00 | 42s792ms609µs873ns | read has no clock uncertainty | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | /cockroach.roachpb.Internal/Batch | 131 |
| 2018-06-04 19:47:52.580672+00:00 | 42s792ms615µs623ns | command queue | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | /cockroach.roachpb.Internal/Batch | 131 |
| 2018-06-04 19:47:52.58068+00:00 | 42s792ms623µs138ns | waiting for read lock | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | /cockroach.roachpb.Internal/Batch | 131 |
| 2018-06-04 19:47:52.580697+00:00 | 42s792ms640µs370ns | read completed | [n1,s1,r50/1:/Table/52/1/2{600-700}] | | /cockroach.roachpb.Internal/Batch | 131 |
| 2018-06-04 19:47:52.580806+00:00 | 42s792ms748µs900ns | === SPAN START: dist sender === | | | dist sender | 137 |
| 2018-06-04 19:47:52.580824+00:00 | 42s792ms767µs650ns | querying next range at /Table/52/1/2700 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 137 |
| 2018-06-04 19:47:52.580855+00:00 | 42s792ms797µs984ns | r51: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 137 |
| 2018-06-04 19:47:52.580872+00:00 | 42s792ms815µs33ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 137 |
| 2018-06-04 19:47:52.580885+00:00 | 42s792ms828µs685ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 138 |
| 2018-06-04 19:47:52.580889+00:00 | 42s792ms832µs335ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 138 |
| 2018-06-04 19:47:52.580894+00:00 | 42s792ms837µs327ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 138 |
| 2018-06-04 19:47:52.58102+00:00 | 42s792ms963µs242ns | read-only path | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | /cockroach.roachpb.Internal/Batch | 138 |
| 2018-06-04 19:47:52.581025+00:00 | 42s792ms968µs218ns | request range lease (attempt #1) | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | /cockroach.roachpb.Internal/Batch | 138 |
| 2018-06-04 19:47:52.581032+00:00 | 42s792ms975µs165ns | === SPAN START: request range lease === | | | request range lease | 139 |
| 2018-06-04 19:47:52.581039+00:00 | 42s792ms982µs314ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 140 |
| | | === | | | | |
| 2018-06-04 19:47:52.581059+00:00 | 42s793ms1µs999ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 141 |
| 2018-06-04 19:47:52.581126+00:00 | 42s793ms69µs163ns | === SPAN START: dist sender === | | | dist sender | 142 |
| 2018-06-04 19:47:52.581137+00:00 | 42s793ms80µs249ns | querying next range at /System/NodeLiveness/1 | [s1,r51/1:/Table/52/1/2{700-800},txn=0d2cc3f8,n1] | | dist sender | 142 |
| 2018-06-04 19:47:52.581159+00:00 | 42s793ms102µs801ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r51/1:/Table/52/1/2{700-800},txn=0d2cc3f8,n1] | | dist sender | 142 |
| 2018-06-04 19:47:52.581175+00:00 | 42s793ms118µs318ns | sending request to alex-laptop.local:26259 | [s1,r51/1:/Table/52/1/2{700-800},txn=0d2cc3f8,n1] | | dist sender | 142 |
| 2018-06-04 19:47:52.581195+00:00 | 42s793ms137µs979ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 143 |
| 2018-06-04 19:47:52.581487+00:00 | 42s793ms430µs299ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 421 |
| 2018-06-04 19:47:52.581514+00:00 | 42s793ms457µs210ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 421 |
| 2018-06-04 19:47:52.581522+00:00 | 42s793ms465µs75ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 421 |
| 2018-06-04 19:47:52.581528+00:00 | 42s793ms471µs413ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 421 |
| 2018-06-04 19:47:52.581541+00:00 | 42s793ms484µs108ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 421 |
| 2018-06-04 19:47:52.581566+00:00 | 42s793ms509µs763ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 421 |
| 2018-06-04 19:47:52.58164+00:00 | 42s793ms582µs850ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 421 |
| 2018-06-04 19:47:52.581643+00:00 | 42s793ms585µs987ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 421 |
| 2018-06-04 19:47:52.582229+00:00 | 42s794ms172µs29ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 421 |
| 2018-06-04 19:47:52.582517+00:00 | 42s794ms459µs852ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 421 |
| 2018-06-04 19:47:52.582538+00:00 | 42s794ms481µs518ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 421 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:47:52.583063+00:00 | 42s795ms6µs476ns | heartbeat 1528141682.081062044,0 | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | liveness heartbeat | 141 |
| 2018-06-04 19:47:52.58307+00:00 | 42s795ms13µs212ns | read-write path | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 140 |
| 2018-06-04 19:47:52.583076+00:00 | 42s795ms19µs679ns | applied timestamp cache | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 140 |
| 2018-06-04 19:47:52.583156+00:00 | 42s795ms98µs992ns | evaluated request | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 140 |
| 2018-06-04 19:47:52.583158+00:00 | 42s795ms101µs479ns | acquired {raft,replica}mu | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 140 |
| 2018-06-04 19:47:55.579404+00:00 | 45s791ms346µs906ns | retry proposal 354f02bfb169152d: reasonTicks | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 140 |
| 2018-06-04 19:47:55.579458+00:00 | 45s791ms401µs25ns | applied timestamp cache | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 140 |
| 2018-06-04 19:47:55.57961+00:00 | 45s791ms553µs241ns | evaluated request | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 140 |
| 2018-06-04 19:47:55.579614+00:00 | 45s791ms557µs592ns | acquired {raft,replica}mu | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 140 |
| 2018-06-04 19:47:56.975657+00:00 | 47s187ms600µs128ns | retry proposal 0e3307c9b66fbc18: reasonNewLeader | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 140 |
| 2018-06-04 19:47:56.975694+00:00 | 47s187ms637µs276ns | applied timestamp cache | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 140 |
| 2018-06-04 19:47:56.975827+00:00 | 47s187ms770µs232ns | evaluated request | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 140 |
| 2018-06-04 19:47:56.975831+00:00 | 47s187ms774µs537ns | acquired {raft,replica}mu | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 140 |
| 2018-06-04 19:47:56.976364+00:00 | 47s188ms307µs658ns | retry proposal 1a4b3e4a157db04d: reasonNewLeaderOrConfigChange | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 140 |
| 2018-06-04 19:47:56.976445+00:00 | 47s188ms388µs727ns | applied timestamp cache | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 140 |
| 2018-06-04 19:47:56.976535+00:00 | 47s188ms478µs411ns | evaluated request | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 140 |
| 2018-06-04 19:47:56.976702+00:00 | 47s188ms644µs948ns | acquired {raft,replica}mu | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 140 |
| 2018-06-04 19:47:56.978081+00:00 | 47s190ms24µs809ns | command proposed from replica (n1,s1):1 with lease #3 incompatible | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 140 |
| | | to repl=(n1,s1):1 seq=4 start=1528141568.366493773,0 epo=3 | | | | |
| | | pro=1528141672.581027426,0 | | | | |
| 2018-06-04 19:47:56.978104+00:00 | 47s190ms47µs105ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 140 |
| | | seq=4 start=1528141568.366493773,0 epo=3 pro=1528141672.581027426,0 | | | | |
| | | with repl=(n1,s1):1 seq=4 start=1528141568.366493773,0 epo=3 | | | | |
| | | pro=1528141672.581027426,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:56.978168+00:00 | 47s190ms111µs12ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=4 | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 140 |
| | | start=1528141568.366493773,0 epo=3 pro=1528141672.581027426,0 | | | | |
| | | with repl=(n1,s1):1 seq=4 start=1528141568.366493773,0 epo=3 | | | | |
| | | pro=1528141672.581027426,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:56.978215+00:00 | 47s190ms158µs327ns | read has no clock uncertainty | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | /cockroach.roachpb.Internal/Batch | 138 |
| 2018-06-04 19:47:56.97822+00:00 | 47s190ms163µs444ns | command queue | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | /cockroach.roachpb.Internal/Batch | 138 |
| 2018-06-04 19:47:56.978228+00:00 | 47s190ms171µs311ns | waiting for read lock | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | /cockroach.roachpb.Internal/Batch | 138 |
| 2018-06-04 19:47:56.97824+00:00 | 47s190ms183µs746ns | read completed | [n1,s1,r51/1:/Table/52/1/2{700-800}] | | /cockroach.roachpb.Internal/Batch | 138 |
| 2018-06-04 19:47:56.978326+00:00 | 47s190ms268µs823ns | === SPAN START: dist sender === | | | dist sender | 144 |
| 2018-06-04 19:47:56.978341+00:00 | 47s190ms284µs50ns | querying next range at /Table/52/1/2800 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 144 |
| 2018-06-04 19:47:56.978374+00:00 | 47s190ms317µs423ns | r52: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 144 |
| 2018-06-04 19:47:56.978397+00:00 | 47s190ms340µs764ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 144 |
| 2018-06-04 19:47:56.978411+00:00 | 47s190ms354µs334ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 145 |
| 2018-06-04 19:47:56.978415+00:00 | 47s190ms358µs52ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 145 |
| 2018-06-04 19:47:56.978421+00:00 | 47s190ms364µs181ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 145 |
| 2018-06-04 19:47:56.978515+00:00 | 47s190ms458µs538ns | read-only path | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | /cockroach.roachpb.Internal/Batch | 145 |
| 2018-06-04 19:47:56.978521+00:00 | 47s190ms464µs463ns | request range lease (attempt #1) | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | /cockroach.roachpb.Internal/Batch | 145 |
| 2018-06-04 19:47:56.978528+00:00 | 47s190ms471µs441ns | === SPAN START: request range lease === | | | request range lease | 146 |
| 2018-06-04 19:47:56.978536+00:00 | 47s190ms479µs225ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 147 |
| | | === | | | | |
| 2018-06-04 19:47:56.978559+00:00 | 47s190ms502µs45ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 148 |
| 2018-06-04 19:47:56.978636+00:00 | 47s190ms579µs557ns | === SPAN START: dist sender === | | | dist sender | 149 |
| 2018-06-04 19:47:56.97865+00:00 | 47s190ms593µs323ns | querying next range at /System/NodeLiveness/1 | [s1,r52/1:/Table/52/1/2{800-900},txn=0efd3681,n1] | | dist sender | 149 |
| 2018-06-04 19:47:56.978674+00:00 | 47s190ms617µs381ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r52/1:/Table/52/1/2{800-900},txn=0efd3681,n1] | | dist sender | 149 |
| 2018-06-04 19:47:56.978693+00:00 | 47s190ms636µs250ns | sending request to alex-laptop.local:26259 | [s1,r52/1:/Table/52/1/2{800-900},txn=0efd3681,n1] | | dist sender | 149 |
| 2018-06-04 19:47:56.978714+00:00 | 47s190ms656µs982ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 150 |
| 2018-06-04 19:47:56.979044+00:00 | 47s190ms987µs473ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 422 |
| 2018-06-04 19:47:56.979082+00:00 | 47s191ms25µs325ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 422 |
| 2018-06-04 19:47:56.979095+00:00 | 47s191ms38µs29ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 422 |
| 2018-06-04 19:47:56.979102+00:00 | 47s191ms45µs696ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 422 |
| 2018-06-04 19:47:56.979118+00:00 | 47s191ms61µs814ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 422 |
| 2018-06-04 19:47:56.979155+00:00 | 47s191ms98µs204ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 422 |
| 2018-06-04 19:47:56.979243+00:00 | 47s191ms185µs866ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 422 |
| 2018-06-04 19:47:56.979249+00:00 | 47s191ms192µs136ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 422 |
| 2018-06-04 19:47:56.979957+00:00 | 47s191ms900µs496ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 422 |
| 2018-06-04 19:47:56.980294+00:00 | 47s192ms237µs269ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 422 |
| 2018-06-04 19:47:56.980319+00:00 | 47s192ms262µs17ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 422 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:47:56.980836+00:00 | 47s192ms778µs928ns | heartbeat 1528141686.478562725,0 | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | liveness heartbeat | 148 |
| 2018-06-04 19:47:56.980847+00:00 | 47s192ms790µs222ns | read-write path | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 147 |
| 2018-06-04 19:47:56.980857+00:00 | 47s192ms800µs23ns | applied timestamp cache | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 147 |
| 2018-06-04 19:47:56.980966+00:00 | 47s192ms908µs850ns | evaluated request | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 147 |
| 2018-06-04 19:47:56.980969+00:00 | 47s192ms911µs986ns | acquired {raft,replica}mu | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 147 |
| 2018-06-04 19:47:59.974274+00:00 | 50s186ms217µs652ns | retry proposal 60047f5d953606bb: reasonTicks | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 147 |
| 2018-06-04 19:47:59.974369+00:00 | 50s186ms312µs278ns | applied timestamp cache | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 147 |
| 2018-06-04 19:47:59.974509+00:00 | 50s186ms452µs602ns | evaluated request | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 147 |
| 2018-06-04 19:47:59.974516+00:00 | 50s186ms458µs825ns | acquired {raft,replica}mu | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 147 |
| 2018-06-04 19:47:59.977904+00:00 | 50s189ms847µs619ns | retry proposal 6e90aac970cf90e2: reasonNewLeader | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 147 |
| 2018-06-04 19:47:59.97797+00:00 | 50s189ms913µs580ns | applied timestamp cache | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 147 |
| 2018-06-04 19:47:59.978078+00:00 | 50s190ms21µs640ns | evaluated request | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 147 |
| 2018-06-04 19:47:59.978094+00:00 | 50s190ms37µs64ns | acquired {raft,replica}mu | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 147 |
| 2018-06-04 19:47:59.978698+00:00 | 50s190ms640µs823ns | retry proposal 48753e6c4b8bede6: reasonNewLeaderOrConfigChange | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 147 |
| 2018-06-04 19:47:59.978747+00:00 | 50s190ms690µs412ns | applied timestamp cache | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 147 |
| 2018-06-04 19:47:59.978818+00:00 | 50s190ms761µs143ns | evaluated request | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 147 |
| 2018-06-04 19:47:59.978825+00:00 | 50s190ms768µs757ns | acquired {raft,replica}mu | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 147 |
| 2018-06-04 19:47:59.979889+00:00 | 50s191ms832µs717ns | command proposed from replica (n1,s1):1 with lease #5 incompatible | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 147 |
| | | to repl=(n1,s1):1 seq=6 start=1528141568.366606927,0 epo=3 | | | | |
| | | pro=1528141676.978523513,0 | | | | |
| 2018-06-04 19:47:59.979917+00:00 | 50s191ms860µs247ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 147 |
| | | seq=6 start=1528141568.366606927,0 epo=3 pro=1528141676.978523513,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.366606927,0 epo=3 | | | | |
| | | pro=1528141676.978523513,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:59.980007+00:00 | 50s191ms950µs403ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=6 | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 147 |
| | | start=1528141568.366606927,0 epo=3 pro=1528141676.978523513,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.366606927,0 epo=3 | | | | |
| | | pro=1528141676.978523513,0: proposed under invalid lease | | | | |
| 2018-06-04 19:47:59.980076+00:00 | 50s192ms18µs989ns | read has no clock uncertainty | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | /cockroach.roachpb.Internal/Batch | 145 |
| 2018-06-04 19:47:59.980083+00:00 | 50s192ms26µs694ns | command queue | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | /cockroach.roachpb.Internal/Batch | 145 |
| 2018-06-04 19:47:59.980093+00:00 | 50s192ms36µs40ns | waiting for read lock | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | /cockroach.roachpb.Internal/Batch | 145 |
| 2018-06-04 19:47:59.980109+00:00 | 50s192ms52µs405ns | read completed | [n1,s1,r52/1:/Table/52/1/2{800-900}] | | /cockroach.roachpb.Internal/Batch | 145 |
| 2018-06-04 19:47:59.980218+00:00 | 50s192ms160µs916ns | === SPAN START: dist sender === | | | dist sender | 151 |
| 2018-06-04 19:47:59.980236+00:00 | 50s192ms178µs821ns | querying next range at /Table/52/1/2900 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 151 |
| 2018-06-04 19:47:59.980265+00:00 | 50s192ms208µs212ns | r53: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 151 |
| 2018-06-04 19:47:59.980289+00:00 | 50s192ms232µs90ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 151 |
| 2018-06-04 19:47:59.980303+00:00 | 50s192ms246µs476ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 152 |
| 2018-06-04 19:47:59.980307+00:00 | 50s192ms250µs354ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 152 |
| 2018-06-04 19:47:59.980313+00:00 | 50s192ms256µs124ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 152 |
| 2018-06-04 19:47:59.980412+00:00 | 50s192ms355µs401ns | read-only path | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | /cockroach.roachpb.Internal/Batch | 152 |
| 2018-06-04 19:47:59.980418+00:00 | 50s192ms360µs948ns | request range lease (attempt #1) | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | /cockroach.roachpb.Internal/Batch | 152 |
| 2018-06-04 19:47:59.980426+00:00 | 50s192ms369µs289ns | === SPAN START: request range lease === | | | request range lease | 153 |
| 2018-06-04 19:47:59.980434+00:00 | 50s192ms377µs85ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 154 |
| | | === | | | | |
| 2018-06-04 19:47:59.980455+00:00 | 50s192ms398µs448ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 155 |
| 2018-06-04 19:47:59.980532+00:00 | 50s192ms474µs896ns | === SPAN START: dist sender === | | | dist sender | 156 |
| 2018-06-04 19:47:59.980545+00:00 | 50s192ms488µs4ns | querying next range at /System/NodeLiveness/1 | [s1,r53/1:/Table/52/1/{2900-3000},txn=98b8aee0,n1] | | dist sender | 156 |
| 2018-06-04 19:47:59.980602+00:00 | 50s192ms545µs117ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r53/1:/Table/52/1/{2900-3000},txn=98b8aee0,n1] | | dist sender | 156 |
| 2018-06-04 19:47:59.980624+00:00 | 50s192ms567µs258ns | sending request to alex-laptop.local:26259 | [s1,r53/1:/Table/52/1/{2900-3000},txn=98b8aee0,n1] | | dist sender | 156 |
| 2018-06-04 19:47:59.980648+00:00 | 50s192ms591µs679ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 157 |
| 2018-06-04 19:47:59.981028+00:00 | 50s192ms971µs186ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 423 |
| 2018-06-04 19:47:59.981071+00:00 | 50s193ms14µs526ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 423 |
| 2018-06-04 19:47:59.981081+00:00 | 50s193ms24µs361ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 423 |
| 2018-06-04 19:47:59.981089+00:00 | 50s193ms31µs859ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 423 |
| 2018-06-04 19:47:59.981103+00:00 | 50s193ms45µs835ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 423 |
| 2018-06-04 19:47:59.981133+00:00 | 50s193ms76µs367ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 423 |
| 2018-06-04 19:47:59.981211+00:00 | 50s193ms153µs978ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 423 |
| 2018-06-04 19:47:59.981214+00:00 | 50s193ms157µs630ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 423 |
| 2018-06-04 19:47:59.981867+00:00 | 50s193ms810µs782ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 423 |
| 2018-06-04 19:47:59.982206+00:00 | 50s194ms149µs363ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 423 |
| 2018-06-04 19:47:59.982232+00:00 | 50s194ms175µs679ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 423 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:47:59.982736+00:00 | 50s194ms679µs524ns | heartbeat 1528141689.480459322,0 | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | liveness heartbeat | 155 |
| 2018-06-04 19:47:59.982744+00:00 | 50s194ms686µs973ns | read-write path | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | [async] storage.pendingLeaseRequest: requesting lease | 154 |
| 2018-06-04 19:47:59.982751+00:00 | 50s194ms694µs219ns | applied timestamp cache | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | [async] storage.pendingLeaseRequest: requesting lease | 154 |
| 2018-06-04 19:47:59.982837+00:00 | 50s194ms780µs638ns | evaluated request | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | [async] storage.pendingLeaseRequest: requesting lease | 154 |
| 2018-06-04 19:47:59.98284+00:00 | 50s194ms783µs534ns | acquired {raft,replica}mu | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | [async] storage.pendingLeaseRequest: requesting lease | 154 |
| 2018-06-04 19:48:02.979186+00:00 | 53s191ms129µs120ns | retry proposal 521d2ff9e203c9c1: reasonTicks | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | [async] storage.pendingLeaseRequest: requesting lease | 154 |
| 2018-06-04 19:48:02.979245+00:00 | 53s191ms188µs465ns | applied timestamp cache | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | [async] storage.pendingLeaseRequest: requesting lease | 154 |
| 2018-06-04 19:48:02.979411+00:00 | 53s191ms354µs600ns | evaluated request | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | [async] storage.pendingLeaseRequest: requesting lease | 154 |
| 2018-06-04 19:48:02.979416+00:00 | 53s191ms359µs442ns | acquired {raft,replica}mu | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | [async] storage.pendingLeaseRequest: requesting lease | 154 |
| 2018-06-04 19:48:03.978627+00:00 | 54s190ms570µs122ns | retry proposal 7c2965dee3e5e9bc: reasonNewLeader | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | [async] storage.pendingLeaseRequest: requesting lease | 154 |
| 2018-06-04 19:48:03.978675+00:00 | 54s190ms618µs645ns | applied timestamp cache | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | [async] storage.pendingLeaseRequest: requesting lease | 154 |
| 2018-06-04 19:48:03.978878+00:00 | 54s190ms820µs906ns | evaluated request | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | [async] storage.pendingLeaseRequest: requesting lease | 154 |
| 2018-06-04 19:48:03.978883+00:00 | 54s190ms825µs906ns | acquired {raft,replica}mu | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | [async] storage.pendingLeaseRequest: requesting lease | 154 |
| 2018-06-04 19:48:03.979553+00:00 | 54s191ms495µs940ns | retry proposal 136923c43ffe19f9: reasonNewLeaderOrConfigChange | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | [async] storage.pendingLeaseRequest: requesting lease | 154 |
| 2018-06-04 19:48:03.979641+00:00 | 54s191ms583µs875ns | applied timestamp cache | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | [async] storage.pendingLeaseRequest: requesting lease | 154 |
| 2018-06-04 19:48:03.979741+00:00 | 54s191ms684µs520ns | evaluated request | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | [async] storage.pendingLeaseRequest: requesting lease | 154 |
| 2018-06-04 19:48:03.97985+00:00 | 54s191ms792µs836ns | acquired {raft,replica}mu | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | [async] storage.pendingLeaseRequest: requesting lease | 154 |
| 2018-06-04 19:48:03.981294+00:00 | 54s193ms237µs744ns | command proposed from replica (n1,s1):1 with lease #5 incompatible | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | [async] storage.pendingLeaseRequest: requesting lease | 154 |
| | | to repl=(n1,s1):1 seq=6 start=1528141568.366723061,0 epo=3 | | | | |
| | | pro=1528141679.980420780,0 | | | | |
| 2018-06-04 19:48:03.981317+00:00 | 54s193ms260µs335ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | [async] storage.pendingLeaseRequest: requesting lease | 154 |
| | | seq=6 start=1528141568.366723061,0 epo=3 pro=1528141679.980420780,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.366723061,0 epo=3 | | | | |
| | | pro=1528141679.980420780,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:03.981393+00:00 | 54s193ms336µs414ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=6 | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | [async] storage.pendingLeaseRequest: requesting lease | 154 |
| | | start=1528141568.366723061,0 epo=3 pro=1528141679.980420780,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.366723061,0 epo=3 | | | | |
| | | pro=1528141679.980420780,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:03.981438+00:00 | 54s193ms381µs376ns | read has no clock uncertainty | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | /cockroach.roachpb.Internal/Batch | 152 |
| 2018-06-04 19:48:03.981444+00:00 | 54s193ms386µs973ns | command queue | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | /cockroach.roachpb.Internal/Batch | 152 |
| 2018-06-04 19:48:03.981451+00:00 | 54s193ms393µs963ns | waiting for read lock | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | /cockroach.roachpb.Internal/Batch | 152 |
| 2018-06-04 19:48:03.981463+00:00 | 54s193ms406µs622ns | read completed | [n1,s1,r53/1:/Table/52/1/{2900-3000}] | | /cockroach.roachpb.Internal/Batch | 152 |
| 2018-06-04 19:48:03.981553+00:00 | 54s193ms496µs292ns | === SPAN START: dist sender === | | | dist sender | 158 |
| 2018-06-04 19:48:03.981569+00:00 | 54s193ms512µs275ns | querying next range at /Table/52/1/3000 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 158 |
| 2018-06-04 19:48:03.981601+00:00 | 54s193ms544µs337ns | r54: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 158 |
| 2018-06-04 19:48:03.981625+00:00 | 54s193ms568µs270ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 158 |
| 2018-06-04 19:48:03.981639+00:00 | 54s193ms582µs158ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 159 |
| 2018-06-04 19:48:03.981643+00:00 | 54s193ms586µs502ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 159 |
| 2018-06-04 19:48:03.981649+00:00 | 54s193ms592µs789ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 159 |
| 2018-06-04 19:48:03.98178+00:00 | 54s193ms723µs76ns | read-only path | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | /cockroach.roachpb.Internal/Batch | 159 |
| 2018-06-04 19:48:03.981788+00:00 | 54s193ms730µs950ns | request range lease (attempt #1) | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | /cockroach.roachpb.Internal/Batch | 159 |
| 2018-06-04 19:48:03.981797+00:00 | 54s193ms740µs753ns | === SPAN START: request range lease === | | | request range lease | 160 |
| 2018-06-04 19:48:03.98181+00:00 | 54s193ms752µs991ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| | | === | | | | |
| 2018-06-04 19:48:03.981837+00:00 | 54s193ms780µs149ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 162 |
| 2018-06-04 19:48:03.981954+00:00 | 54s193ms897µs810ns | === SPAN START: dist sender === | | | dist sender | 163 |
| 2018-06-04 19:48:03.98199+00:00 | 54s193ms933µs296ns | querying next range at /System/NodeLiveness/1 | [s1,r54/1:/Table/52/1/3{000-100},txn=2ef2a1d3,n1] | | dist sender | 163 |
| 2018-06-04 19:48:03.98204+00:00 | 54s193ms982µs870ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r54/1:/Table/52/1/3{000-100},txn=2ef2a1d3,n1] | | dist sender | 163 |
| 2018-06-04 19:48:03.982063+00:00 | 54s194ms6µs814ns | sending request to alex-laptop.local:26259 | [s1,r54/1:/Table/52/1/3{000-100},txn=2ef2a1d3,n1] | | dist sender | 163 |
| 2018-06-04 19:48:03.982087+00:00 | 54s194ms30µs196ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 164 |
| 2018-06-04 19:48:03.982369+00:00 | 54s194ms311µs970ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 424 |
| 2018-06-04 19:48:03.982398+00:00 | 54s194ms341µs629ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 424 |
| 2018-06-04 19:48:03.982408+00:00 | 54s194ms351µs133ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 424 |
| 2018-06-04 19:48:03.982415+00:00 | 54s194ms358µs760ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 424 |
| 2018-06-04 19:48:03.982429+00:00 | 54s194ms372µs600ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 424 |
| 2018-06-04 19:48:03.98246+00:00 | 54s194ms402µs845ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 424 |
| 2018-06-04 19:48:03.982533+00:00 | 54s194ms476µs705ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 424 |
| 2018-06-04 19:48:03.982537+00:00 | 54s194ms480µs728ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 424 |
| 2018-06-04 19:48:03.983209+00:00 | 54s195ms152µs799ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 424 |
| 2018-06-04 19:48:03.98355+00:00 | 54s195ms493µs421ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 424 |
| 2018-06-04 19:48:03.983576+00:00 | 54s195ms518µs819ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 424 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:48:03.984127+00:00 | 54s196ms70µs510ns | heartbeat 1528141693.481842115,0 | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | liveness heartbeat | 162 |
| 2018-06-04 19:48:03.984136+00:00 | 54s196ms79µs259ns | read-write path | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:03.984144+00:00 | 54s196ms87µs183ns | applied timestamp cache | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:03.984223+00:00 | 54s196ms166µs33ns | evaluated request | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:03.984226+00:00 | 54s196ms168µs913ns | acquired {raft,replica}mu | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:06.977506+00:00 | 57s189ms449µs106ns | retry proposal 32deadedcbce0388: reasonTicks | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:06.977566+00:00 | 57s189ms508µs918ns | applied timestamp cache | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:06.977725+00:00 | 57s189ms668µs580ns | evaluated request | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:06.97773+00:00 | 57s189ms673µs240ns | acquired {raft,replica}mu | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.375614+00:00 | 57s587ms556µs968ns | retry proposal 09e268d06f1c3bb8: reasonNewLeader | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.375662+00:00 | 57s587ms605µs584ns | applied timestamp cache | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.375805+00:00 | 57s587ms748µs736ns | evaluated request | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.37581+00:00 | 57s587ms753µs125ns | acquired {raft,replica}mu | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.376261+00:00 | 57s588ms204µs734ns | retry proposal 7b1ee072c7df699c: reasonNewLeaderOrConfigChange | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.376296+00:00 | 57s588ms239µs219ns | applied timestamp cache | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.376403+00:00 | 57s588ms346µs283ns | evaluated request | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.376584+00:00 | 57s588ms527µs138ns | acquired {raft,replica}mu | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.377701+00:00 | 57s589ms644µs545ns | retry proposal 12caad6c99cbca5e: reasonNewLeader | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.377717+00:00 | 57s589ms660µs275ns | applied timestamp cache | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.377779+00:00 | 57s589ms722µs599ns | evaluated request | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.377782+00:00 | 57s589ms724µs879ns | acquired {raft,replica}mu | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.378459+00:00 | 57s590ms401µs950ns | retry proposal 5158536746cb0696: reasonNewLeader | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.378527+00:00 | 57s590ms470µs791ns | applied timestamp cache | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.37865+00:00 | 57s590ms593µs287ns | evaluated request | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.378668+00:00 | 57s590ms610µs827ns | acquired {raft,replica}mu | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.379128+00:00 | 57s591ms71µs105ns | retry proposal 41efe8b35aad54a8: reasonNewLeaderOrConfigChange | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.379195+00:00 | 57s591ms138µs399ns | applied timestamp cache | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.379278+00:00 | 57s591ms221µs743ns | evaluated request | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.379287+00:00 | 57s591ms230µs710ns | acquired {raft,replica}mu | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| 2018-06-04 19:48:07.379987+00:00 | 57s591ms930µs567ns | command proposed from replica (n1,s1):1 with lease #5 incompatible | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| | | to repl=(n1,s1):1 seq=6 start=1528141568.366844055,0 epo=3 | | | | |
| | | pro=1528141683.981790853,0 | | | | |
| 2018-06-04 19:48:07.38001+00:00 | 57s591ms953µs345ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| | | seq=6 start=1528141568.366844055,0 epo=3 pro=1528141683.981790853,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.366844055,0 epo=3 | | | | |
| | | pro=1528141683.981790853,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:07.380136+00:00 | 57s592ms79µs485ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=6 | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | [async] storage.pendingLeaseRequest: requesting lease | 161 |
| | | start=1528141568.366844055,0 epo=3 pro=1528141683.981790853,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.366844055,0 epo=3 | | | | |
| | | pro=1528141683.981790853,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:07.380249+00:00 | 57s592ms191µs890ns | read has no clock uncertainty | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | /cockroach.roachpb.Internal/Batch | 159 |
| 2018-06-04 19:48:07.380265+00:00 | 57s592ms208µs158ns | command queue | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | /cockroach.roachpb.Internal/Batch | 159 |
| 2018-06-04 19:48:07.380276+00:00 | 57s592ms219µs626ns | waiting for read lock | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | /cockroach.roachpb.Internal/Batch | 159 |
| 2018-06-04 19:48:07.380317+00:00 | 57s592ms260µs444ns | read completed | [n1,s1,r54/1:/Table/52/1/3{000-100}] | | /cockroach.roachpb.Internal/Batch | 159 |
| 2018-06-04 19:48:07.380404+00:00 | 57s592ms346µs961ns | === SPAN START: dist sender === | | | dist sender | 165 |
| 2018-06-04 19:48:07.380422+00:00 | 57s592ms365µs404ns | querying next range at /Table/52/1/3100 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 165 |
| 2018-06-04 19:48:07.380451+00:00 | 57s592ms394µs698ns | r55: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 165 |
| 2018-06-04 19:48:07.380474+00:00 | 57s592ms416µs946ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 165 |
| 2018-06-04 19:48:07.380488+00:00 | 57s592ms431µs122ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 166 |
| 2018-06-04 19:48:07.380492+00:00 | 57s592ms435µs202ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 166 |
| 2018-06-04 19:48:07.380497+00:00 | 57s592ms440µs276ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 166 |
| 2018-06-04 19:48:07.3806+00:00 | 57s592ms543µs570ns | read-only path | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | /cockroach.roachpb.Internal/Batch | 166 |
| 2018-06-04 19:48:07.380608+00:00 | 57s592ms551µs718ns | request range lease (attempt #1) | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | /cockroach.roachpb.Internal/Batch | 166 |
| 2018-06-04 19:48:07.380615+00:00 | 57s592ms558µs377ns | === SPAN START: request range lease === | | | request range lease | 167 |
| 2018-06-04 19:48:07.380623+00:00 | 57s592ms566µs526ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 168 |
| | | === | | | | |
| 2018-06-04 19:48:07.380646+00:00 | 57s592ms589µs432ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 169 |
| 2018-06-04 19:48:07.380723+00:00 | 57s592ms666µs712ns | === SPAN START: dist sender === | | | dist sender | 170 |
| 2018-06-04 19:48:07.380738+00:00 | 57s592ms681µs127ns | querying next range at /System/NodeLiveness/1 | [s1,r55/1:/Table/52/1/3{100-200},txn=e940293a,n1] | | dist sender | 170 |
| 2018-06-04 19:48:07.380763+00:00 | 57s592ms706µs171ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r55/1:/Table/52/1/3{100-200},txn=e940293a,n1] | | dist sender | 170 |
| 2018-06-04 19:48:07.380781+00:00 | 57s592ms724µs155ns | sending request to alex-laptop.local:26259 | [s1,r55/1:/Table/52/1/3{100-200},txn=e940293a,n1] | | dist sender | 170 |
| 2018-06-04 19:48:07.3808+00:00 | 57s592ms743µs775ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 171 |
| 2018-06-04 19:48:07.381094+00:00 | 57s593ms37µs82ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 425 |
| 2018-06-04 19:48:07.381122+00:00 | 57s593ms65µs512ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 425 |
| 2018-06-04 19:48:07.38113+00:00 | 57s593ms73µs492ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 425 |
| 2018-06-04 19:48:07.381136+00:00 | 57s593ms79µs573ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 425 |
| 2018-06-04 19:48:07.381148+00:00 | 57s593ms91µs484ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 425 |
| 2018-06-04 19:48:07.38118+00:00 | 57s593ms122µs838ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 425 |
| 2018-06-04 19:48:07.381245+00:00 | 57s593ms187µs978ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 425 |
| 2018-06-04 19:48:07.381247+00:00 | 57s593ms190µs766ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 425 |
| 2018-06-04 19:48:07.381874+00:00 | 57s593ms817µs282ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 425 |
| 2018-06-04 19:48:07.382195+00:00 | 57s594ms138µs111ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 425 |
| 2018-06-04 19:48:07.382216+00:00 | 57s594ms159µs281ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 425 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:48:07.382664+00:00 | 57s594ms607µs734ns | heartbeat 1528141696.880650569,0 | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | liveness heartbeat | 169 |
| 2018-06-04 19:48:07.382675+00:00 | 57s594ms618µs131ns | read-write path | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 168 |
| 2018-06-04 19:48:07.382681+00:00 | 57s594ms624µs447ns | applied timestamp cache | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 168 |
| 2018-06-04 19:48:07.382762+00:00 | 57s594ms704µs875ns | evaluated request | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 168 |
| 2018-06-04 19:48:07.382764+00:00 | 57s594ms707µs487ns | acquired {raft,replica}mu | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 168 |
| 2018-06-04 19:48:10.374687+00:00 | 1m586ms630µs91ns | retry proposal 1741b3ffc8de467a: reasonTicks | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 168 |
| 2018-06-04 19:48:10.374908+00:00 | 1m586ms850µs932ns | applied timestamp cache | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 168 |
| 2018-06-04 19:48:10.375054+00:00 | 1m586ms996µs932ns | evaluated request | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 168 |
| 2018-06-04 19:48:10.375058+00:00 | 1m587ms1µs740ns | acquired {raft,replica}mu | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 168 |
| 2018-06-04 19:48:10.981492+00:00 | 1m1s193ms435µs154ns | retry proposal 1731e3ceb868bc28: reasonNewLeader | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 168 |
| 2018-06-04 19:48:10.981543+00:00 | 1m1s193ms485µs824ns | applied timestamp cache | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 168 |
| 2018-06-04 19:48:10.981645+00:00 | 1m1s193ms587µs872ns | evaluated request | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 168 |
| 2018-06-04 19:48:10.981655+00:00 | 1m1s193ms598µs128ns | acquired {raft,replica}mu | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 168 |
| 2018-06-04 19:48:10.981922+00:00 | 1m1s193ms865µs701ns | retry proposal 354f61f461f985fc: reasonNewLeaderOrConfigChange | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 168 |
| 2018-06-04 19:48:10.981959+00:00 | 1m1s193ms902µs5ns | applied timestamp cache | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 168 |
| 2018-06-04 19:48:10.98208+00:00 | 1m1s194ms23µs410ns | evaluated request | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 168 |
| 2018-06-04 19:48:10.982479+00:00 | 1m1s194ms422µs258ns | acquired {raft,replica}mu | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 168 |
| 2018-06-04 19:48:10.982932+00:00 | 1m1s194ms874µs935ns | command proposed from replica (n1,s1):1 with lease #3 incompatible | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 168 |
| | | to repl=(n1,s1):1 seq=4 start=1528141568.366962026,0 epo=3 | | | | |
| | | pro=1528141687.380611076,0 | | | | |
| 2018-06-04 19:48:10.98295+00:00 | 1m1s194ms893µs443ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 168 |
| | | seq=4 start=1528141568.366962026,0 epo=3 pro=1528141687.380611076,0 | | | | |
| | | with repl=(n1,s1):1 seq=4 start=1528141568.366962026,0 epo=3 | | | | |
| | | pro=1528141687.380611076,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:10.983018+00:00 | 1m1s194ms961µs26ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=4 | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | [async] storage.pendingLeaseRequest: requesting lease | 168 |
| | | start=1528141568.366962026,0 epo=3 pro=1528141687.380611076,0 | | | | |
| | | with repl=(n1,s1):1 seq=4 start=1528141568.366962026,0 epo=3 | | | | |
| | | pro=1528141687.380611076,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:10.983065+00:00 | 1m1s195ms8µs284ns | read has no clock uncertainty | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | /cockroach.roachpb.Internal/Batch | 166 |
| 2018-06-04 19:48:10.98307+00:00 | 1m1s195ms13µs782ns | command queue | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | /cockroach.roachpb.Internal/Batch | 166 |
| 2018-06-04 19:48:10.983078+00:00 | 1m1s195ms21µs403ns | waiting for read lock | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | /cockroach.roachpb.Internal/Batch | 166 |
| 2018-06-04 19:48:10.983092+00:00 | 1m1s195ms35µs229ns | read completed | [n1,s1,r55/1:/Table/52/1/3{100-200}] | | /cockroach.roachpb.Internal/Batch | 166 |
| 2018-06-04 19:48:10.983189+00:00 | 1m1s195ms132µs314ns | === SPAN START: dist sender === | | | dist sender | 172 |
| 2018-06-04 19:48:10.983221+00:00 | 1m1s195ms164µs463ns | querying next range at /Table/52/1/3200 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 172 |
| 2018-06-04 19:48:10.983251+00:00 | 1m1s195ms194µs752ns | r56: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 172 |
| 2018-06-04 19:48:10.983269+00:00 | 1m1s195ms212µs354ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 172 |
| 2018-06-04 19:48:10.983281+00:00 | 1m1s195ms224µs462ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 173 |
| 2018-06-04 19:48:10.983284+00:00 | 1m1s195ms227µs536ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 173 |
| 2018-06-04 19:48:10.983289+00:00 | 1m1s195ms231µs980ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 173 |
| 2018-06-04 19:48:10.983384+00:00 | 1m1s195ms327µs643ns | read-only path | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | /cockroach.roachpb.Internal/Batch | 173 |
| 2018-06-04 19:48:10.983389+00:00 | 1m1s195ms331µs945ns | request range lease (attempt #1) | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | /cockroach.roachpb.Internal/Batch | 173 |
| 2018-06-04 19:48:10.983396+00:00 | 1m1s195ms339µs486ns | === SPAN START: request range lease === | | | request range lease | 174 |
| 2018-06-04 19:48:10.983405+00:00 | 1m1s195ms348µs365ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 175 |
| | | === | | | | |
| 2018-06-04 19:48:10.983426+00:00 | 1m1s195ms368µs987ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 176 |
| 2018-06-04 19:48:10.983498+00:00 | 1m1s195ms441µs225ns | === SPAN START: dist sender === | | | dist sender | 177 |
| 2018-06-04 19:48:10.983508+00:00 | 1m1s195ms451µs330ns | querying next range at /System/NodeLiveness/1 | [s1,r56/1:/Table/52/1/3{200-300},txn=99a38b5a,n1] | | dist sender | 177 |
| 2018-06-04 19:48:10.983529+00:00 | 1m1s195ms471µs862ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r56/1:/Table/52/1/3{200-300},txn=99a38b5a,n1] | | dist sender | 177 |
| 2018-06-04 19:48:10.983543+00:00 | 1m1s195ms486µs70ns | sending request to alex-laptop.local:26259 | [s1,r56/1:/Table/52/1/3{200-300},txn=99a38b5a,n1] | | dist sender | 177 |
| 2018-06-04 19:48:10.983559+00:00 | 1m1s195ms502µs582ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 178 |
| 2018-06-04 19:48:10.983807+00:00 | 1m1s195ms750µs386ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 426 |
| 2018-06-04 19:48:10.983827+00:00 | 1m1s195ms770µs401ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 426 |
| 2018-06-04 19:48:10.983834+00:00 | 1m1s195ms776µs821ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 426 |
| 2018-06-04 19:48:10.983839+00:00 | 1m1s195ms781µs954ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 426 |
| 2018-06-04 19:48:10.983849+00:00 | 1m1s195ms791µs967ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 426 |
| 2018-06-04 19:48:10.983863+00:00 | 1m1s195ms806µs351ns | === SPAN START: request range lease === | | | request range lease | 427 |
| 2018-06-04 19:48:10.983867+00:00 | 1m1s195ms810µs540ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 428 |
| | | === | | | | |
| 2018-06-04 19:48:10.983895+00:00 | 1m1s195ms838µs180ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 426 |
| 2018-06-04 19:48:10.983936+00:00 | 1m1s195ms879µs619ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | [async] storage.pendingLeaseRequest: requesting lease | 428 |
| 2018-06-04 19:48:10.983949+00:00 | 1m1s195ms892µs383ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | [async] storage.pendingLeaseRequest: requesting lease | 428 |
| 2018-06-04 19:48:10.98397+00:00 | 1m1s195ms913µs108ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 426 |
| 2018-06-04 19:48:10.983972+00:00 | 1m1s195ms915µs500ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 426 |
| 2018-06-04 19:48:10.984028+00:00 | 1m1s195ms971µs527ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | [async] storage.pendingLeaseRequest: requesting lease | 428 |
| 2018-06-04 19:48:10.984178+00:00 | 1m1s196ms121µs431ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | [async] storage.pendingLeaseRequest: requesting lease | 428 |
| 2018-06-04 19:48:10.984497+00:00 | 1m1s196ms439µs867ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 426 |
| 2018-06-04 19:48:10.984773+00:00 | 1m1s196ms716µs278ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 426 |
| 2018-06-04 19:48:10.984792+00:00 | 1m1s196ms734µs819ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 426 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:48:10.985329+00:00 | 1m1s197ms271µs908ns | heartbeat 1528141700.483430326,0 | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | liveness heartbeat | 176 |
| 2018-06-04 19:48:10.985334+00:00 | 1m1s197ms277µs420ns | read-write path | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 175 |
| 2018-06-04 19:48:10.985339+00:00 | 1m1s197ms282µs758ns | applied timestamp cache | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 175 |
| 2018-06-04 19:48:10.985402+00:00 | 1m1s197ms345µs430ns | evaluated request | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 175 |
| 2018-06-04 19:48:10.985404+00:00 | 1m1s197ms347µs687ns | acquired {raft,replica}mu | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 175 |
| 2018-06-04 19:48:13.973608+00:00 | 1m4s185ms551µs280ns | retry proposal 29ebbddfecdf5014: reasonTicks | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 175 |
| 2018-06-04 19:48:13.9738+00:00 | 1m4s185ms743µs121ns | applied timestamp cache | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 175 |
| 2018-06-04 19:48:13.973956+00:00 | 1m4s185ms899µs502ns | evaluated request | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 175 |
| 2018-06-04 19:48:13.973961+00:00 | 1m4s185ms904µs554ns | acquired {raft,replica}mu | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 175 |
| 2018-06-04 19:48:13.977609+00:00 | 1m4s189ms552µs729ns | retry proposal 0d6a3e3c365df2dc: reasonNewLeader | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 175 |
| 2018-06-04 19:48:13.977637+00:00 | 1m4s189ms580µs274ns | applied timestamp cache | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 175 |
| 2018-06-04 19:48:13.977745+00:00 | 1m4s189ms687µs854ns | evaluated request | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 175 |
| 2018-06-04 19:48:13.977748+00:00 | 1m4s189ms691µs295ns | acquired {raft,replica}mu | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 175 |
| 2018-06-04 19:48:13.978793+00:00 | 1m4s190ms736µs324ns | retry proposal 6a3b9b259baf82a6: reasonNewLeaderOrConfigChange | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 175 |
| 2018-06-04 19:48:13.978824+00:00 | 1m4s190ms767µs810ns | applied timestamp cache | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 175 |
| 2018-06-04 19:48:13.978919+00:00 | 1m4s190ms862µs409ns | evaluated request | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 175 |
| 2018-06-04 19:48:13.978923+00:00 | 1m4s190ms866µs639ns | acquired {raft,replica}mu | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 175 |
| 2018-06-04 19:48:13.980096+00:00 | 1m4s192ms39µs713ns | command proposed from replica (n1,s1):1 with lease #5 incompatible | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 175 |
| | | to repl=(n1,s1):1 seq=6 start=1528141568.367080934,0 epo=3 | | | | |
| | | pro=1528141690.983390542,0 | | | | |
| 2018-06-04 19:48:13.980114+00:00 | 1m4s192ms57µs146ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 175 |
| | | seq=6 start=1528141568.367080934,0 epo=3 pro=1528141690.983390542,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.367080934,0 epo=3 | | | | |
| | | pro=1528141690.983390542,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:13.980198+00:00 | 1m4s192ms141µs547ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=6 | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 175 |
| | | start=1528141568.367080934,0 epo=3 pro=1528141690.983390542,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.367080934,0 epo=3 | | | | |
| | | pro=1528141690.983390542,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:13.980263+00:00 | 1m4s192ms205µs829ns | read has no clock uncertainty | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | /cockroach.roachpb.Internal/Batch | 173 |
| 2018-06-04 19:48:13.98027+00:00 | 1m4s192ms213µs424ns | command queue | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | /cockroach.roachpb.Internal/Batch | 173 |
| 2018-06-04 19:48:13.980279+00:00 | 1m4s192ms222µs210ns | waiting for read lock | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | /cockroach.roachpb.Internal/Batch | 173 |
| 2018-06-04 19:48:13.980291+00:00 | 1m4s192ms234µs390ns | read completed | [n1,s1,r56/1:/Table/52/1/3{200-300}] | | /cockroach.roachpb.Internal/Batch | 173 |
| 2018-06-04 19:48:13.98038+00:00 | 1m4s192ms322µs835ns | === SPAN START: dist sender === | | | dist sender | 179 |
| 2018-06-04 19:48:13.980396+00:00 | 1m4s192ms338µs911ns | querying next range at /Table/52/1/3300 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 179 |
| 2018-06-04 19:48:13.980427+00:00 | 1m4s192ms370µs683ns | r57: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 179 |
| 2018-06-04 19:48:13.98045+00:00 | 1m4s192ms393µs320ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 179 |
| 2018-06-04 19:48:13.980465+00:00 | 1m4s192ms408µs96ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 180 |
| 2018-06-04 19:48:13.980469+00:00 | 1m4s192ms412µs207ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 180 |
| 2018-06-04 19:48:13.980475+00:00 | 1m4s192ms418µs164ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 180 |
| 2018-06-04 19:48:13.98058+00:00 | 1m4s192ms522µs817ns | read-only path | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | /cockroach.roachpb.Internal/Batch | 180 |
| 2018-06-04 19:48:13.980586+00:00 | 1m4s192ms529µs329ns | request range lease (attempt #1) | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | /cockroach.roachpb.Internal/Batch | 180 |
| 2018-06-04 19:48:13.980593+00:00 | 1m4s192ms536µs654ns | === SPAN START: request range lease === | | | request range lease | 181 |
| 2018-06-04 19:48:13.980602+00:00 | 1m4s192ms545µs73ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 182 |
| | | === | | | | |
| 2018-06-04 19:48:13.980625+00:00 | 1m4s192ms568µs774ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 183 |
| 2018-06-04 19:48:13.980717+00:00 | 1m4s192ms659µs982ns | === SPAN START: dist sender === | | | dist sender | 184 |
| 2018-06-04 19:48:13.980735+00:00 | 1m4s192ms678µs403ns | querying next range at /System/NodeLiveness/1 | [s1,r57/1:/Table/52/1/3{300-400},txn=87c7c651,n1] | | dist sender | 184 |
| 2018-06-04 19:48:13.980771+00:00 | 1m4s192ms713µs908ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r57/1:/Table/52/1/3{300-400},txn=87c7c651,n1] | | dist sender | 184 |
| 2018-06-04 19:48:13.980798+00:00 | 1m4s192ms741µs655ns | sending request to alex-laptop.local:26259 | [s1,r57/1:/Table/52/1/3{300-400},txn=87c7c651,n1] | | dist sender | 184 |
| 2018-06-04 19:48:13.980826+00:00 | 1m4s192ms769µs499ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 185 |
| 2018-06-04 19:48:13.981103+00:00 | 1m4s193ms46µs64ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 429 |
| 2018-06-04 19:48:13.981133+00:00 | 1m4s193ms76µs404ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 429 |
| 2018-06-04 19:48:13.981141+00:00 | 1m4s193ms84µs654ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 429 |
| 2018-06-04 19:48:13.981148+00:00 | 1m4s193ms91µs25ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 429 |
| 2018-06-04 19:48:13.981161+00:00 | 1m4s193ms104µs725ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 429 |
| 2018-06-04 19:48:13.981192+00:00 | 1m4s193ms134µs833ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 429 |
| 2018-06-04 19:48:13.981263+00:00 | 1m4s193ms206µs701ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 429 |
| 2018-06-04 19:48:13.981267+00:00 | 1m4s193ms209µs874ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 429 |
| 2018-06-04 19:48:13.981916+00:00 | 1m4s193ms858µs895ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 429 |
| 2018-06-04 19:48:13.98223+00:00 | 1m4s194ms172µs883ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 429 |
| 2018-06-04 19:48:13.982255+00:00 | 1m4s194ms197µs838ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 429 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:48:13.982789+00:00 | 1m4s194ms731µs998ns | heartbeat 1528141703.480631068,0 | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | liveness heartbeat | 183 |
| 2018-06-04 19:48:13.982797+00:00 | 1m4s194ms740µs340ns | read-write path | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 182 |
| 2018-06-04 19:48:13.982804+00:00 | 1m4s194ms746µs909ns | applied timestamp cache | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 182 |
| 2018-06-04 19:48:13.982887+00:00 | 1m4s194ms830µs600ns | evaluated request | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 182 |
| 2018-06-04 19:48:13.98289+00:00 | 1m4s194ms833µs695ns | acquired {raft,replica}mu | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 182 |
| 2018-06-04 19:48:16.974521+00:00 | 1m7s186ms464µs773ns | retry proposal 1c4bff7e5b080dc8: reasonTicks | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 182 |
| 2018-06-04 19:48:16.974651+00:00 | 1m7s186ms594µs32ns | applied timestamp cache | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 182 |
| 2018-06-04 19:48:16.974928+00:00 | 1m7s186ms871µs284ns | evaluated request | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 182 |
| 2018-06-04 19:48:16.974936+00:00 | 1m7s186ms878µs828ns | acquired {raft,replica}mu | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 182 |
| 2018-06-04 19:48:16.97675+00:00 | 1m7s188ms692µs970ns | retry proposal 5b4f522c3faa3ed3: reasonNewLeader | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 182 |
| 2018-06-04 19:48:16.976775+00:00 | 1m7s188ms718µs394ns | applied timestamp cache | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 182 |
| 2018-06-04 19:48:16.976886+00:00 | 1m7s188ms829µs302ns | evaluated request | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 182 |
| 2018-06-04 19:48:16.97689+00:00 | 1m7s188ms833µs115ns | acquired {raft,replica}mu | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 182 |
| 2018-06-04 19:48:16.977336+00:00 | 1m7s189ms279µs93ns | retry proposal 137e316a319ed617: reasonNewLeaderOrConfigChange | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 182 |
| 2018-06-04 19:48:16.97736+00:00 | 1m7s189ms303µs806ns | applied timestamp cache | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 182 |
| 2018-06-04 19:48:16.977466+00:00 | 1m7s189ms408µs989ns | evaluated request | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 182 |
| 2018-06-04 19:48:16.977716+00:00 | 1m7s189ms659µs274ns | acquired {raft,replica}mu | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 182 |
| 2018-06-04 19:48:16.979107+00:00 | 1m7s191ms50µs89ns | command proposed from replica (n1,s1):1 with lease #5 incompatible | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 182 |
| | | to repl=(n1,s1):1 seq=6 start=1528141568.367198971,0 epo=3 | | | | |
| | | pro=1528141693.980588737,0 | | | | |
| 2018-06-04 19:48:16.979133+00:00 | 1m7s191ms76µs422ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 182 |
| | | seq=6 start=1528141568.367198971,0 epo=3 pro=1528141693.980588737,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.367198971,0 epo=3 | | | | |
| | | pro=1528141693.980588737,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:16.979207+00:00 | 1m7s191ms150µs509ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=6 | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | [async] storage.pendingLeaseRequest: requesting lease | 182 |
| | | start=1528141568.367198971,0 epo=3 pro=1528141693.980588737,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.367198971,0 epo=3 | | | | |
| | | pro=1528141693.980588737,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:16.979264+00:00 | 1m7s191ms207µs335ns | read has no clock uncertainty | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | /cockroach.roachpb.Internal/Batch | 180 |
| 2018-06-04 19:48:16.97927+00:00 | 1m7s191ms213µs183ns | command queue | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | /cockroach.roachpb.Internal/Batch | 180 |
| 2018-06-04 19:48:16.979278+00:00 | 1m7s191ms221µs783ns | waiting for read lock | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | /cockroach.roachpb.Internal/Batch | 180 |
| 2018-06-04 19:48:16.979297+00:00 | 1m7s191ms240µs442ns | read completed | [n1,s1,r57/1:/Table/52/1/3{300-400}] | | /cockroach.roachpb.Internal/Batch | 180 |
| 2018-06-04 19:48:16.979388+00:00 | 1m7s191ms331µs403ns | === SPAN START: dist sender === | | | dist sender | 186 |
| 2018-06-04 19:48:16.979404+00:00 | 1m7s191ms347µs459ns | querying next range at /Table/52/1/3400 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 186 |
| 2018-06-04 19:48:16.979436+00:00 | 1m7s191ms379µs137ns | r58: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 186 |
| 2018-06-04 19:48:16.97946+00:00 | 1m7s191ms403µs365ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 186 |
| 2018-06-04 19:48:16.979475+00:00 | 1m7s191ms417µs918ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 187 |
| 2018-06-04 19:48:16.979478+00:00 | 1m7s191ms421µs569ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 187 |
| 2018-06-04 19:48:16.979484+00:00 | 1m7s191ms426µs955ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 187 |
| 2018-06-04 19:48:16.979579+00:00 | 1m7s191ms522µs354ns | read-only path | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | /cockroach.roachpb.Internal/Batch | 187 |
| 2018-06-04 19:48:16.979584+00:00 | 1m7s191ms527µs599ns | request range lease (attempt #1) | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | /cockroach.roachpb.Internal/Batch | 187 |
| 2018-06-04 19:48:16.979591+00:00 | 1m7s191ms534µs273ns | === SPAN START: request range lease === | | | request range lease | 188 |
| 2018-06-04 19:48:16.979599+00:00 | 1m7s191ms542µs640ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 189 |
| | | === | | | | |
| 2018-06-04 19:48:16.979623+00:00 | 1m7s191ms566µs805ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 190 |
| 2018-06-04 19:48:16.979696+00:00 | 1m7s191ms639µs32ns | === SPAN START: dist sender === | | | dist sender | 191 |
| 2018-06-04 19:48:16.979708+00:00 | 1m7s191ms651µs394ns | querying next range at /System/NodeLiveness/1 | [s1,r58/1:/Table/52/1/3{400-500},txn=ec5df6e3,n1] | | dist sender | 191 |
| 2018-06-04 19:48:16.979732+00:00 | 1m7s191ms675µs598ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r58/1:/Table/52/1/3{400-500},txn=ec5df6e3,n1] | | dist sender | 191 |
| 2018-06-04 19:48:16.97975+00:00 | 1m7s191ms693µs416ns | sending request to alex-laptop.local:26259 | [s1,r58/1:/Table/52/1/3{400-500},txn=ec5df6e3,n1] | | dist sender | 191 |
| 2018-06-04 19:48:16.97977+00:00 | 1m7s191ms713µs373ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 192 |
| 2018-06-04 19:48:16.980135+00:00 | 1m7s192ms78µs670ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 430 |
| 2018-06-04 19:48:16.980186+00:00 | 1m7s192ms129µs622ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 430 |
| 2018-06-04 19:48:16.980203+00:00 | 1m7s192ms146µs266ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 430 |
| 2018-06-04 19:48:16.980223+00:00 | 1m7s192ms166µs791ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 430 |
| 2018-06-04 19:48:16.980247+00:00 | 1m7s192ms190µs568ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 430 |
| 2018-06-04 19:48:16.98029+00:00 | 1m7s192ms233µs651ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 430 |
| 2018-06-04 19:48:16.980379+00:00 | 1m7s192ms322µs712ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 430 |
| 2018-06-04 19:48:16.980383+00:00 | 1m7s192ms326µs477ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 430 |
| 2018-06-04 19:48:16.981071+00:00 | 1m7s193ms14µs249ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 430 |
| 2018-06-04 19:48:16.98143+00:00 | 1m7s193ms373µs188ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 430 |
| 2018-06-04 19:48:16.981458+00:00 | 1m7s193ms400µs872ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 430 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:48:16.982082+00:00 | 1m7s194ms25µs479ns | heartbeat 1528141706.479627712,0 | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | liveness heartbeat | 190 |
| 2018-06-04 19:48:16.982106+00:00 | 1m7s194ms49µs185ns | read-write path | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 189 |
| 2018-06-04 19:48:16.982118+00:00 | 1m7s194ms60µs847ns | applied timestamp cache | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 189 |
| 2018-06-04 19:48:16.98223+00:00 | 1m7s194ms173µs531ns | evaluated request | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 189 |
| 2018-06-04 19:48:16.982236+00:00 | 1m7s194ms179µs315ns | acquired {raft,replica}mu | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 189 |
| 2018-06-04 19:48:19.975584+00:00 | 1m10s187ms527µs752ns | retry proposal 09ee2cb5c03bfc69: reasonTicks | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 189 |
| 2018-06-04 19:48:19.975626+00:00 | 1m10s187ms569µs503ns | applied timestamp cache | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 189 |
| 2018-06-04 19:48:19.975762+00:00 | 1m10s187ms705µs370ns | evaluated request | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 189 |
| 2018-06-04 19:48:19.975765+00:00 | 1m10s187ms708µs807ns | acquired {raft,replica}mu | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 189 |
| 2018-06-04 19:48:20.978642+00:00 | 1m11s190ms585µs767ns | retry proposal 52a298fb4b9f0ded: reasonNewLeader | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 189 |
| 2018-06-04 19:48:20.978686+00:00 | 1m11s190ms628µs923ns | applied timestamp cache | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 189 |
| 2018-06-04 19:48:20.978831+00:00 | 1m11s190ms774µs570ns | evaluated request | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 189 |
| 2018-06-04 19:48:20.978837+00:00 | 1m11s190ms780µs12ns | acquired {raft,replica}mu | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 189 |
| 2018-06-04 19:48:20.979666+00:00 | 1m11s191ms609µs300ns | retry proposal 7d768542ae6d53d3: reasonNewLeaderOrConfigChange | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 189 |
| 2018-06-04 19:48:20.97973+00:00 | 1m11s191ms673µs364ns | applied timestamp cache | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 189 |
| 2018-06-04 19:48:20.979872+00:00 | 1m11s191ms815µs201ns | evaluated request | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 189 |
| 2018-06-04 19:48:20.979946+00:00 | 1m11s191ms889µs528ns | acquired {raft,replica}mu | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 189 |
| 2018-06-04 19:48:20.981183+00:00 | 1m11s193ms126µs184ns | command proposed from replica (n1,s1):1 with lease #5 incompatible | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 189 |
| | | to repl=(n1,s1):1 seq=6 start=1528141568.367317917,0 epo=3 | | | | |
| | | pro=1528141696.979586898,0 | | | | |
| 2018-06-04 19:48:20.98121+00:00 | 1m11s193ms152µs877ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 189 |
| | | seq=6 start=1528141568.367317917,0 epo=3 pro=1528141696.979586898,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.367317917,0 epo=3 | | | | |
| | | pro=1528141696.979586898,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:20.981294+00:00 | 1m11s193ms237µs399ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=6 | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | [async] storage.pendingLeaseRequest: requesting lease | 189 |
| | | start=1528141568.367317917,0 epo=3 pro=1528141696.979586898,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.367317917,0 epo=3 | | | | |
| | | pro=1528141696.979586898,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:20.981351+00:00 | 1m11s193ms294µs751ns | read has no clock uncertainty | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | /cockroach.roachpb.Internal/Batch | 187 |
| 2018-06-04 19:48:20.981357+00:00 | 1m11s193ms300µs263ns | command queue | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | /cockroach.roachpb.Internal/Batch | 187 |
| 2018-06-04 19:48:20.981365+00:00 | 1m11s193ms308µs95ns | waiting for read lock | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | /cockroach.roachpb.Internal/Batch | 187 |
| 2018-06-04 19:48:20.981377+00:00 | 1m11s193ms320µs542ns | read completed | [n1,s1,r58/1:/Table/52/1/3{400-500}] | | /cockroach.roachpb.Internal/Batch | 187 |
| 2018-06-04 19:48:20.981464+00:00 | 1m11s193ms407µs750ns | === SPAN START: dist sender === | | | dist sender | 193 |
| 2018-06-04 19:48:20.981483+00:00 | 1m11s193ms426µs565ns | querying next range at /Table/52/1/3500 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 193 |
| 2018-06-04 19:48:20.981516+00:00 | 1m11s193ms459µs418ns | === SPAN START: range lookup === | | | range lookup | 194 |
| 2018-06-04 19:48:20.981543+00:00 | 1m11s193ms485µs836ns | === SPAN START: dist sender === | | | dist sender | 195 |
| 2018-06-04 19:48:20.981572+00:00 | 1m11s193ms515µs633ns | querying next range at /Meta2/Table/52/1/3500/NULL | [client=[::1]:57501,user=root,txn=9352587e,range-lookup=/Table/52/1/3500,n1] | | dist sender | 195 |
| 2018-06-04 19:48:20.981591+00:00 | 1m11s193ms534µs754ns | === SPAN START: range lookup === | | | range lookup | 196 |
| 2018-06-04 19:48:20.981608+00:00 | 1m11s193ms551µs313ns | === SPAN START: dist sender === | | | dist sender | 197 |
| 2018-06-04 19:48:20.981641+00:00 | 1m11s193ms584µs657ns | querying next range at /Meta1/Table/52/1/3500/NULL/NULL | [client=[::1]:57501,user=root,txn=9352587e,range-lookup=/Table/52/1/3500,range-lookup=/Meta2/Table/52/1/3500/NULL,n1] | | dist sender | 197 |
| 2018-06-04 19:48:20.981659+00:00 | 1m11s193ms602µs182ns | === SPAN START: range lookup === | | | range lookup | 198 |
| 2018-06-04 19:48:20.981701+00:00 | 1m11s193ms644µs781ns | looked up range descriptor | [client=[::1]:57501,user=root,txn=9352587e,range-lookup=/Table/52/1/3500,range-lookup=/Meta2/Table/52/1/3500/NULL,n1] | | dist sender | 197 |
| 2018-06-04 19:48:20.981736+00:00 | 1m11s193ms679µs796ns | r1: sending batch 1 Scan to (n2,s2):3 | [client=[::1]:57501,user=root,txn=9352587e,range-lookup=/Table/52/1/3500,range-lookup=/Meta2/Table/52/1/3500/NULL,n1] | | dist sender | 197 |
| 2018-06-04 19:48:20.981794+00:00 | 1m11s193ms737µs314ns | sending request to alex-laptop.local:26259 | [client=[::1]:57501,user=root,txn=9352587e,range-lookup=/Table/52/1/3500,range-lookup=/Meta2/Table/52/1/3500/NULL,n1] | | dist sender | 197 |
| 2018-06-04 19:48:20.981806+00:00 | 1m11s193ms749µs112ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 199 |
| 2018-06-04 19:48:20.982141+00:00 | 1m11s194ms84µs47ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 431 |
| 2018-06-04 19:48:20.982172+00:00 | 1m11s194ms114µs926ns | 1 Scan | [n2] | | /cockroach.roachpb.Internal/Batch | 431 |
| 2018-06-04 19:48:20.982179+00:00 | 1m11s194ms122µs356ns | executing 1 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 431 |
| 2018-06-04 19:48:20.982186+00:00 | 1m11s194ms129µs605ns | read-only path | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 431 |
| 2018-06-04 19:48:20.982195+00:00 | 1m11s194ms138µs799ns | command queue | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 431 |
| 2018-06-04 19:48:20.982197+00:00 | 1m11s194ms140µs348ns | operation accepts inconsistent results | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 431 |
| 2018-06-04 19:48:20.982199+00:00 | 1m11s194ms142µs623ns | waiting for read lock | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 431 |
| 2018-06-04 19:48:20.982472+00:00 | 1m11s194ms414µs995ns | engine stats: {InternalDeleteSkippedCount:2 TimeBoundNumSSTs:0} | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 431 |
| 2018-06-04 19:48:20.982478+00:00 | 1m11s194ms420µs920ns | read completed | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 431 |
| 2018-06-04 19:48:20.982761+00:00 | 1m11s194ms704µs60ns | looked up range descriptor | [client=[::1]:57501,user=root,txn=9352587e,range-lookup=/Table/52/1/3500,n1] | | dist sender | 195 |
| 2018-06-04 19:48:20.982787+00:00 | 1m11s194ms730µs85ns | r1: sending batch 1 Scan to (n2,s2):3 | [client=[::1]:57501,user=root,txn=9352587e,range-lookup=/Table/52/1/3500,n1] | | dist sender | 195 |
| 2018-06-04 19:48:20.982824+00:00 | 1m11s194ms766µs966ns | sending request to alex-laptop.local:26259 | [client=[::1]:57501,user=root,txn=9352587e,range-lookup=/Table/52/1/3500,n1] | | dist sender | 195 |
| 2018-06-04 19:48:20.982833+00:00 | 1m11s194ms776µs94ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 200 |
| 2018-06-04 19:48:20.983049+00:00 | 1m11s194ms991µs871ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 432 |
| 2018-06-04 19:48:20.983074+00:00 | 1m11s195ms17µs562ns | 1 Scan | [n2] | | /cockroach.roachpb.Internal/Batch | 432 |
| 2018-06-04 19:48:20.983081+00:00 | 1m11s195ms23µs882ns | executing 1 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 432 |
| 2018-06-04 19:48:20.983086+00:00 | 1m11s195ms29µs506ns | read-only path | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 432 |
| 2018-06-04 19:48:20.983093+00:00 | 1m11s195ms35µs846ns | command queue | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 432 |
| 2018-06-04 19:48:20.983094+00:00 | 1m11s195ms37µs757ns | operation accepts inconsistent results | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 432 |
| 2018-06-04 19:48:20.983097+00:00 | 1m11s195ms40µs439ns | waiting for read lock | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 432 |
| 2018-06-04 19:48:20.983437+00:00 | 1m11s195ms380µs790ns | engine stats: {InternalDeleteSkippedCount:10 TimeBoundNumSSTs:0} | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 432 |
| 2018-06-04 19:48:20.983444+00:00 | 1m11s195ms387µs119ns | read completed | [n2,s2,r1/3:/{Min-System/}] | | /cockroach.roachpb.Internal/Batch | 432 |
| 2018-06-04 19:48:20.983754+00:00 | 1m11s195ms696µs946ns | looked up range descriptor | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 193 |
| 2018-06-04 19:48:20.983778+00:00 | 1m11s195ms721µs311ns | r59: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 193 |
| 2018-06-04 19:48:20.983796+00:00 | 1m11s195ms739µs430ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 193 |
| 2018-06-04 19:48:20.983808+00:00 | 1m11s195ms751µs728ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 201 |
| 2018-06-04 19:48:20.983814+00:00 | 1m11s195ms757µs151ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 201 |
| 2018-06-04 19:48:20.983819+00:00 | 1m11s195ms762µs231ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 201 |
| 2018-06-04 19:48:20.983917+00:00 | 1m11s195ms860µs55ns | read-only path | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | /cockroach.roachpb.Internal/Batch | 201 |
| 2018-06-04 19:48:20.983922+00:00 | 1m11s195ms865µs43ns | request range lease (attempt #1) | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | /cockroach.roachpb.Internal/Batch | 201 |
| 2018-06-04 19:48:20.983928+00:00 | 1m11s195ms870µs954ns | === SPAN START: request range lease === | | | request range lease | 202 |
| 2018-06-04 19:48:20.983934+00:00 | 1m11s195ms877µs393ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 203 |
| | | === | | | | |
| 2018-06-04 19:48:20.983955+00:00 | 1m11s195ms897µs968ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 204 |
| 2018-06-04 19:48:20.984025+00:00 | 1m11s195ms967µs845ns | === SPAN START: dist sender === | | | dist sender | 205 |
| 2018-06-04 19:48:20.984037+00:00 | 1m11s195ms980µs703ns | querying next range at /System/NodeLiveness/1 | [s1,r59/1:/Table/52/1/3{500-600},txn=4a4ee9ce,n1] | | dist sender | 205 |
| 2018-06-04 19:48:20.984059+00:00 | 1m11s196ms2µs335ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r59/1:/Table/52/1/3{500-600},txn=4a4ee9ce,n1] | | dist sender | 205 |
| 2018-06-04 19:48:20.984076+00:00 | 1m11s196ms18µs994ns | sending request to alex-laptop.local:26259 | [s1,r59/1:/Table/52/1/3{500-600},txn=4a4ee9ce,n1] | | dist sender | 205 |
| 2018-06-04 19:48:20.98409+00:00 | 1m11s196ms33µs794ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 206 |
| 2018-06-04 19:48:20.984287+00:00 | 1m11s196ms230µs281ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 433 |
| 2018-06-04 19:48:20.98431+00:00 | 1m11s196ms252µs930ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 433 |
| 2018-06-04 19:48:20.984317+00:00 | 1m11s196ms260µs42ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 433 |
| 2018-06-04 19:48:20.984323+00:00 | 1m11s196ms266µs17ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 433 |
| 2018-06-04 19:48:20.984334+00:00 | 1m11s196ms277µs200ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 433 |
| 2018-06-04 19:48:20.984361+00:00 | 1m11s196ms303µs875ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 433 |
| 2018-06-04 19:48:20.984426+00:00 | 1m11s196ms368µs989ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 433 |
| 2018-06-04 19:48:20.984429+00:00 | 1m11s196ms372µs447ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 433 |
| 2018-06-04 19:48:20.985005+00:00 | 1m11s196ms948µs170ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 433 |
| 2018-06-04 19:48:20.985262+00:00 | 1m11s197ms205µs86ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 433 |
| 2018-06-04 19:48:20.985284+00:00 | 1m11s197ms226µs934ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 433 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:48:20.985768+00:00 | 1m11s197ms711µs57ns | heartbeat 1528141710.483958150,0 | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | liveness heartbeat | 204 |
| 2018-06-04 19:48:20.985775+00:00 | 1m11s197ms718µs89ns | read-write path | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 203 |
| 2018-06-04 19:48:20.985781+00:00 | 1m11s197ms724µs401ns | applied timestamp cache | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 203 |
| 2018-06-04 19:48:20.985855+00:00 | 1m11s197ms798µs374ns | evaluated request | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 203 |
| 2018-06-04 19:48:20.985858+00:00 | 1m11s197ms801µs356ns | acquired {raft,replica}mu | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 203 |
| 2018-06-04 19:48:20.987314+00:00 | 1m11s199ms257µs592ns | retry proposal 231e9504c2f2829e: reasonNewLeader | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 203 |
| 2018-06-04 19:48:20.987332+00:00 | 1m11s199ms275µs793ns | applied timestamp cache | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 203 |
| 2018-06-04 19:48:20.98738+00:00 | 1m11s199ms323µs806ns | evaluated request | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 203 |
| 2018-06-04 19:48:20.98739+00:00 | 1m11s199ms333µs599ns | acquired {raft,replica}mu | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 203 |
| 2018-06-04 19:48:20.987755+00:00 | 1m11s199ms698µs96ns | retry proposal 29e9808d2a9d5ee2: reasonNewLeaderOrConfigChange | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 203 |
| 2018-06-04 19:48:20.987782+00:00 | 1m11s199ms725µs456ns | applied timestamp cache | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 203 |
| 2018-06-04 19:48:20.987834+00:00 | 1m11s199ms777µs256ns | evaluated request | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 203 |
| 2018-06-04 19:48:20.98784+00:00 | 1m11s199ms783µs725ns | acquired {raft,replica}mu | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 203 |
| 2018-06-04 19:48:20.98856+00:00 | 1m11s200ms503µs179ns | command proposed from replica (n1,s1):1 with lease #3 incompatible | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 203 |
| | | to repl=(n1,s1):1 seq=4 start=1528141568.367447288,0 epo=3 | | | | |
| | | pro=1528141700.983924211,0 | | | | |
| 2018-06-04 19:48:20.988578+00:00 | 1m11s200ms521µs129ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 203 |
| | | seq=4 start=1528141568.367447288,0 epo=3 pro=1528141700.983924211,0 | | | | |
| | | with repl=(n1,s1):1 seq=4 start=1528141568.367447288,0 epo=3 | | | | |
| | | pro=1528141700.983924211,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:20.988656+00:00 | 1m11s200ms599µs673ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=4 | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | [async] storage.pendingLeaseRequest: requesting lease | 203 |
| | | start=1528141568.367447288,0 epo=3 pro=1528141700.983924211,0 | | | | |
| | | with repl=(n1,s1):1 seq=4 start=1528141568.367447288,0 epo=3 | | | | |
| | | pro=1528141700.983924211,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:20.988734+00:00 | 1m11s200ms677µs575ns | read has no clock uncertainty | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | /cockroach.roachpb.Internal/Batch | 201 |
| 2018-06-04 19:48:20.98874+00:00 | 1m11s200ms683µs686ns | command queue | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | /cockroach.roachpb.Internal/Batch | 201 |
| 2018-06-04 19:48:20.988746+00:00 | 1m11s200ms689µs714ns | waiting for read lock | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | /cockroach.roachpb.Internal/Batch | 201 |
| 2018-06-04 19:48:20.988759+00:00 | 1m11s200ms702µs87ns | read completed | [n1,s1,r59/1:/Table/52/1/3{500-600}] | | /cockroach.roachpb.Internal/Batch | 201 |
| 2018-06-04 19:48:20.988826+00:00 | 1m11s200ms769µs806ns | === SPAN START: dist sender === | | | dist sender | 207 |
| 2018-06-04 19:48:20.988838+00:00 | 1m11s200ms781µs30ns | querying next range at /Table/52/1/3600 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 207 |
| 2018-06-04 19:48:20.988856+00:00 | 1m11s200ms799µs716ns | r60: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 207 |
| 2018-06-04 19:48:20.988875+00:00 | 1m11s200ms818µs472ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 207 |
| 2018-06-04 19:48:20.988887+00:00 | 1m11s200ms830µs789ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 208 |
| 2018-06-04 19:48:20.988891+00:00 | 1m11s200ms834µs37ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 208 |
| 2018-06-04 19:48:20.988895+00:00 | 1m11s200ms838µs65ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 208 |
| 2018-06-04 19:48:20.988969+00:00 | 1m11s200ms911µs902ns | read-only path | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | /cockroach.roachpb.Internal/Batch | 208 |
| 2018-06-04 19:48:20.988973+00:00 | 1m11s200ms916µs375ns | request range lease (attempt #1) | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | /cockroach.roachpb.Internal/Batch | 208 |
| 2018-06-04 19:48:20.988978+00:00 | 1m11s200ms920µs848ns | === SPAN START: request range lease === | | | request range lease | 209 |
| 2018-06-04 19:48:20.988983+00:00 | 1m11s200ms926µs359ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 210 |
| | | === | | | | |
| 2018-06-04 19:48:20.988999+00:00 | 1m11s200ms941µs862ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 211 |
| 2018-06-04 19:48:20.989049+00:00 | 1m11s200ms992µs166ns | === SPAN START: dist sender === | | | dist sender | 212 |
| 2018-06-04 19:48:20.989061+00:00 | 1m11s201ms4µs85ns | querying next range at /System/NodeLiveness/1 | [s1,r60/1:/Table/52/1/3{600-700},txn=5871726f,n1] | | dist sender | 212 |
| 2018-06-04 19:48:20.989079+00:00 | 1m11s201ms22µs78ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r60/1:/Table/52/1/3{600-700},txn=5871726f,n1] | | dist sender | 212 |
| 2018-06-04 19:48:20.989093+00:00 | 1m11s201ms35µs989ns | sending request to alex-laptop.local:26259 | [s1,r60/1:/Table/52/1/3{600-700},txn=5871726f,n1] | | dist sender | 212 |
| 2018-06-04 19:48:20.989105+00:00 | 1m11s201ms48µs133ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 213 |
| 2018-06-04 19:48:20.989331+00:00 | 1m11s201ms274µs353ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 434 |
| 2018-06-04 19:48:20.989362+00:00 | 1m11s201ms305µs235ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 434 |
| 2018-06-04 19:48:20.989382+00:00 | 1m11s201ms325µs352ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 434 |
| 2018-06-04 19:48:20.989402+00:00 | 1m11s201ms345µs648ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 434 |
| 2018-06-04 19:48:20.989413+00:00 | 1m11s201ms356µs653ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 434 |
| 2018-06-04 19:48:20.989436+00:00 | 1m11s201ms379µs23ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 434 |
| 2018-06-04 19:48:20.989484+00:00 | 1m11s201ms427µs180ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 434 |
| 2018-06-04 19:48:20.989487+00:00 | 1m11s201ms430µs143ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 434 |
| 2018-06-04 19:48:20.989948+00:00 | 1m11s201ms891µs321ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 434 |
| 2018-06-04 19:48:20.99017+00:00 | 1m11s202ms112µs950ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 434 |
| 2018-06-04 19:48:20.990186+00:00 | 1m11s202ms129µs511ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 434 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:48:20.990521+00:00 | 1m11s202ms464µs467ns | heartbeat 1528141710.489001610,0 | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | liveness heartbeat | 211 |
| 2018-06-04 19:48:20.990527+00:00 | 1m11s202ms470µs271ns | read-write path | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 210 |
| 2018-06-04 19:48:20.990532+00:00 | 1m11s202ms475µs143ns | applied timestamp cache | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 210 |
| 2018-06-04 19:48:20.990588+00:00 | 1m11s202ms531µs94ns | evaluated request | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 210 |
| 2018-06-04 19:48:20.99059+00:00 | 1m11s202ms533µs344ns | acquired {raft,replica}mu | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 210 |
| 2018-06-04 19:48:20.991771+00:00 | 1m11s203ms714µs647ns | retry proposal 0c33d4b59cc0a19d: reasonNewLeader | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 210 |
| 2018-06-04 19:48:20.991789+00:00 | 1m11s203ms731µs957ns | applied timestamp cache | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 210 |
| 2018-06-04 19:48:20.991838+00:00 | 1m11s203ms781µs700ns | evaluated request | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 210 |
| 2018-06-04 19:48:20.991845+00:00 | 1m11s203ms787µs838ns | acquired {raft,replica}mu | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 210 |
| 2018-06-04 19:48:20.99217+00:00 | 1m11s204ms113µs96ns | retry proposal 1b598d2dac57d46e: reasonNewLeaderOrConfigChange | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 210 |
| 2018-06-04 19:48:20.992196+00:00 | 1m11s204ms139µs660ns | applied timestamp cache | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 210 |
| 2018-06-04 19:48:20.992246+00:00 | 1m11s204ms188µs846ns | evaluated request | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 210 |
| 2018-06-04 19:48:20.992253+00:00 | 1m11s204ms196µs327ns | acquired {raft,replica}mu | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 210 |
| 2018-06-04 19:48:20.992922+00:00 | 1m11s204ms865µs679ns | command proposed from replica (n1,s1):1 with lease #5 incompatible | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 210 |
| | | to repl=(n1,s1):1 seq=6 start=1528141568.367567054,0 epo=3 | | | | |
| | | pro=1528141700.988974905,0 | | | | |
| 2018-06-04 19:48:20.992943+00:00 | 1m11s204ms886µs225ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 210 |
| | | seq=6 start=1528141568.367567054,0 epo=3 pro=1528141700.988974905,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.367567054,0 epo=3 | | | | |
| | | pro=1528141700.988974905,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:20.992993+00:00 | 1m11s204ms936µs641ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=6 | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | [async] storage.pendingLeaseRequest: requesting lease | 210 |
| | | start=1528141568.367567054,0 epo=3 pro=1528141700.988974905,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.367567054,0 epo=3 | | | | |
| | | pro=1528141700.988974905,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:20.993021+00:00 | 1m11s204ms964µs635ns | read has no clock uncertainty | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | /cockroach.roachpb.Internal/Batch | 208 |
| 2018-06-04 19:48:20.993025+00:00 | 1m11s204ms968µs240ns | command queue | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | /cockroach.roachpb.Internal/Batch | 208 |
| 2018-06-04 19:48:20.99303+00:00 | 1m11s204ms972µs845ns | waiting for read lock | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | /cockroach.roachpb.Internal/Batch | 208 |
| 2018-06-04 19:48:20.993041+00:00 | 1m11s204ms984µs152ns | read completed | [n1,s1,r60/1:/Table/52/1/3{600-700}] | | /cockroach.roachpb.Internal/Batch | 208 |
| 2018-06-04 19:48:20.993091+00:00 | 1m11s205ms34µs253ns | === SPAN START: dist sender === | | | dist sender | 214 |
| 2018-06-04 19:48:20.993104+00:00 | 1m11s205ms47µs322ns | querying next range at /Table/52/1/3700 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 214 |
| 2018-06-04 19:48:20.993129+00:00 | 1m11s205ms71µs857ns | r61: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 214 |
| 2018-06-04 19:48:20.993141+00:00 | 1m11s205ms83µs917ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 214 |
| 2018-06-04 19:48:20.99315+00:00 | 1m11s205ms93µs461ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 215 |
| 2018-06-04 19:48:20.993154+00:00 | 1m11s205ms96µs853ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 215 |
| 2018-06-04 19:48:20.993158+00:00 | 1m11s205ms101µs272ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 215 |
| 2018-06-04 19:48:20.993219+00:00 | 1m11s205ms162µs646ns | read-only path | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | /cockroach.roachpb.Internal/Batch | 215 |
| 2018-06-04 19:48:20.993223+00:00 | 1m11s205ms166µs360ns | request range lease (attempt #1) | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | /cockroach.roachpb.Internal/Batch | 215 |
| 2018-06-04 19:48:20.993227+00:00 | 1m11s205ms170µs740ns | === SPAN START: request range lease === | | | request range lease | 216 |
| 2018-06-04 19:48:20.993232+00:00 | 1m11s205ms175µs477ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| | | === | | | | |
| 2018-06-04 19:48:20.993247+00:00 | 1m11s205ms190µs304ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 218 |
| 2018-06-04 19:48:20.993295+00:00 | 1m11s205ms237µs945ns | === SPAN START: dist sender === | | | dist sender | 219 |
| 2018-06-04 19:48:20.993304+00:00 | 1m11s205ms246µs936ns | querying next range at /System/NodeLiveness/1 | [s1,r61/1:/Table/52/1/3{700-800},txn=18b67d2e,n1] | | dist sender | 219 |
| 2018-06-04 19:48:20.993338+00:00 | 1m11s205ms280µs867ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r61/1:/Table/52/1/3{700-800},txn=18b67d2e,n1] | | dist sender | 219 |
| 2018-06-04 19:48:20.99335+00:00 | 1m11s205ms293µs144ns | sending request to alex-laptop.local:26259 | [s1,r61/1:/Table/52/1/3{700-800},txn=18b67d2e,n1] | | dist sender | 219 |
| 2018-06-04 19:48:20.993362+00:00 | 1m11s205ms305µs521ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 220 |
| 2018-06-04 19:48:20.993529+00:00 | 1m11s205ms472µs710ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 435 |
| 2018-06-04 19:48:20.99356+00:00 | 1m11s205ms502µs941ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 435 |
| 2018-06-04 19:48:20.993566+00:00 | 1m11s205ms509µs406ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 435 |
| 2018-06-04 19:48:20.99357+00:00 | 1m11s205ms513µs770ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 435 |
| 2018-06-04 19:48:20.993578+00:00 | 1m11s205ms521µs751ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 435 |
| 2018-06-04 19:48:20.993598+00:00 | 1m11s205ms541µs514ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 435 |
| 2018-06-04 19:48:20.993642+00:00 | 1m11s205ms585µs144ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 435 |
| 2018-06-04 19:48:20.993644+00:00 | 1m11s205ms587µs624ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 435 |
| 2018-06-04 19:48:20.994068+00:00 | 1m11s206ms11µs270ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 435 |
| 2018-06-04 19:48:20.994309+00:00 | 1m11s206ms252µs340ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 435 |
| 2018-06-04 19:48:20.994327+00:00 | 1m11s206ms270µs41ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 435 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:48:20.994689+00:00 | 1m11s206ms632µs283ns | heartbeat 1528141710.493249798,0 | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | liveness heartbeat | 218 |
| 2018-06-04 19:48:20.994694+00:00 | 1m11s206ms637µs375ns | read-write path | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:20.994709+00:00 | 1m11s206ms651µs989ns | applied timestamp cache | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:20.994761+00:00 | 1m11s206ms704µs288ns | evaluated request | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:20.994763+00:00 | 1m11s206ms706µs393ns | acquired {raft,replica}mu | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:23.973458+00:00 | 1m14s185ms400µs882ns | retry proposal 066c03c114c0716c: reasonTicks | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:23.973505+00:00 | 1m14s185ms448µs628ns | applied timestamp cache | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:23.973642+00:00 | 1m14s185ms585µs511ns | evaluated request | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:23.973646+00:00 | 1m14s185ms589µs502ns | acquired {raft,replica}mu | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.17499+00:00 | 1m14s386ms933µs175ns | retry proposal 6c42e51fc5529ace: reasonNewLeader | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.175026+00:00 | 1m14s386ms969µs300ns | applied timestamp cache | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.175165+00:00 | 1m14s387ms108µs519ns | evaluated request | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.17517+00:00 | 1m14s387ms113µs235ns | acquired {raft,replica}mu | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.175809+00:00 | 1m14s387ms752µs504ns | retry proposal 01ea8736ff466cf1: reasonNewLeaderOrConfigChange | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.175873+00:00 | 1m14s387ms816µs698ns | applied timestamp cache | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.175973+00:00 | 1m14s387ms916µs442ns | evaluated request | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.176031+00:00 | 1m14s387ms974µs385ns | acquired {raft,replica}mu | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.177034+00:00 | 1m14s388ms977µs173ns | retry proposal 2f24eb68b91f89d7: reasonNewLeader | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.177051+00:00 | 1m14s388ms994µs208ns | applied timestamp cache | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.177127+00:00 | 1m14s389ms70µs45ns | evaluated request | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.17713+00:00 | 1m14s389ms73µs378ns | acquired {raft,replica}mu | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.177733+00:00 | 1m14s389ms676µs623ns | retry proposal 297738324b0ba26e: reasonNewLeader | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.177821+00:00 | 1m14s389ms764µs722ns | applied timestamp cache | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.177939+00:00 | 1m14s389ms882µs477ns | evaluated request | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.177951+00:00 | 1m14s389ms894µs548ns | acquired {raft,replica}mu | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.178419+00:00 | 1m14s390ms362µs319ns | retry proposal 7b051c553073cc5f: reasonNewLeaderOrConfigChange | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.17845+00:00 | 1m14s390ms393µs504ns | applied timestamp cache | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.178512+00:00 | 1m14s390ms455µs315ns | evaluated request | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.17852+00:00 | 1m14s390ms463µs11ns | acquired {raft,replica}mu | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| 2018-06-04 19:48:24.179323+00:00 | 1m14s391ms266µs614ns | command proposed from replica (n1,s1):1 with lease #3 incompatible | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| | | to repl=(n1,s1):1 seq=4 start=1528141568.367682122,0 epo=3 | | | | |
| | | pro=1528141700.993224760,0 | | | | |
| 2018-06-04 19:48:24.179348+00:00 | 1m14s391ms291µs321ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| | | seq=4 start=1528141568.367682122,0 epo=3 pro=1528141700.993224760,0 | | | | |
| | | with repl=(n1,s1):1 seq=4 start=1528141568.367682122,0 epo=3 | | | | |
| | | pro=1528141700.993224760,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:24.179455+00:00 | 1m14s391ms398µs673ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=4 | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 217 |
| | | start=1528141568.367682122,0 epo=3 pro=1528141700.993224760,0 | | | | |
| | | with repl=(n1,s1):1 seq=4 start=1528141568.367682122,0 epo=3 | | | | |
| | | pro=1528141700.993224760,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:24.1795+00:00 | 1m14s391ms443µs767ns | read has no clock uncertainty | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | /cockroach.roachpb.Internal/Batch | 215 |
| 2018-06-04 19:48:24.179506+00:00 | 1m14s391ms449µs696ns | command queue | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | /cockroach.roachpb.Internal/Batch | 215 |
| 2018-06-04 19:48:24.179517+00:00 | 1m14s391ms459µs880ns | waiting for read lock | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | /cockroach.roachpb.Internal/Batch | 215 |
| 2018-06-04 19:48:24.179532+00:00 | 1m14s391ms475µs194ns | read completed | [n1,s1,r61/1:/Table/52/1/3{700-800}] | | /cockroach.roachpb.Internal/Batch | 215 |
| 2018-06-04 19:48:24.179624+00:00 | 1m14s391ms567µs160ns | === SPAN START: dist sender === | | | dist sender | 221 |
| 2018-06-04 19:48:24.179637+00:00 | 1m14s391ms580µs577ns | querying next range at /Table/52/1/3800 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 221 |
| 2018-06-04 19:48:24.179668+00:00 | 1m14s391ms611µs185ns | r62: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 221 |
| 2018-06-04 19:48:24.179688+00:00 | 1m14s391ms631µs544ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 221 |
| 2018-06-04 19:48:24.179702+00:00 | 1m14s391ms645µs274ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 222 |
| 2018-06-04 19:48:24.179706+00:00 | 1m14s391ms649µs44ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 222 |
| 2018-06-04 19:48:24.179711+00:00 | 1m14s391ms654µs604ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 222 |
| 2018-06-04 19:48:24.1798+00:00 | 1m14s391ms743µs256ns | read-only path | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | /cockroach.roachpb.Internal/Batch | 222 |
| 2018-06-04 19:48:24.179805+00:00 | 1m14s391ms748µs327ns | request range lease (attempt #1) | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | /cockroach.roachpb.Internal/Batch | 222 |
| 2018-06-04 19:48:24.179811+00:00 | 1m14s391ms754µs522ns | === SPAN START: request range lease === | | | request range lease | 223 |
| 2018-06-04 19:48:24.179821+00:00 | 1m14s391ms763µs951ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 224 |
| | | === | | | | |
| 2018-06-04 19:48:24.179839+00:00 | 1m14s391ms782µs533ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 225 |
| 2018-06-04 19:48:24.179911+00:00 | 1m14s391ms854µs63ns | === SPAN START: dist sender === | | | dist sender | 226 |
| 2018-06-04 19:48:24.179922+00:00 | 1m14s391ms865µs640ns | querying next range at /System/NodeLiveness/1 | [s1,r62/1:/Table/52/1/3{800-900},txn=dd882069,n1] | | dist sender | 226 |
| 2018-06-04 19:48:24.179944+00:00 | 1m14s391ms887µs325ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r62/1:/Table/52/1/3{800-900},txn=dd882069,n1] | | dist sender | 226 |
| 2018-06-04 19:48:24.179961+00:00 | 1m14s391ms904µs17ns | sending request to alex-laptop.local:26259 | [s1,r62/1:/Table/52/1/3{800-900},txn=dd882069,n1] | | dist sender | 226 |
| 2018-06-04 19:48:24.179982+00:00 | 1m14s391ms925µs186ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 227 |
| 2018-06-04 19:48:24.180229+00:00 | 1m14s392ms172µs662ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 436 |
| 2018-06-04 19:48:24.180254+00:00 | 1m14s392ms197µs493ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 436 |
| 2018-06-04 19:48:24.180261+00:00 | 1m14s392ms204µs769ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 436 |
| 2018-06-04 19:48:24.180268+00:00 | 1m14s392ms210µs927ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 436 |
| 2018-06-04 19:48:24.18028+00:00 | 1m14s392ms223µs152ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 436 |
| 2018-06-04 19:48:24.18031+00:00 | 1m14s392ms253µs188ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 436 |
| 2018-06-04 19:48:24.180373+00:00 | 1m14s392ms316µs453ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 436 |
| 2018-06-04 19:48:24.180376+00:00 | 1m14s392ms319µs658ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 436 |
| 2018-06-04 19:48:24.180976+00:00 | 1m14s392ms918µs998ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 436 |
| 2018-06-04 19:48:24.181244+00:00 | 1m14s393ms187µs651ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 436 |
| 2018-06-04 19:48:24.181265+00:00 | 1m14s393ms208µs426ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 436 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:48:24.181726+00:00 | 1m14s393ms668µs971ns | heartbeat 1528141713.679844278,0 | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | liveness heartbeat | 225 |
| 2018-06-04 19:48:24.181734+00:00 | 1m14s393ms676µs862ns | read-write path | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 224 |
| 2018-06-04 19:48:24.181742+00:00 | 1m14s393ms685µs501ns | applied timestamp cache | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 224 |
| 2018-06-04 19:48:24.181843+00:00 | 1m14s393ms786µs573ns | evaluated request | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 224 |
| 2018-06-04 19:48:24.181846+00:00 | 1m14s393ms789µs402ns | acquired {raft,replica}mu | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 224 |
| 2018-06-04 19:48:27.176094+00:00 | 1m17s388ms37µs104ns | retry proposal 4d10dddb489fb624: reasonTicks | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 224 |
| 2018-06-04 19:48:27.176139+00:00 | 1m17s388ms82µs122ns | applied timestamp cache | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 224 |
| 2018-06-04 19:48:27.176286+00:00 | 1m17s388ms229µs179ns | evaluated request | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 224 |
| 2018-06-04 19:48:27.17629+00:00 | 1m17s388ms233µs585ns | acquired {raft,replica}mu | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 224 |
| 2018-06-04 19:48:28.577983+00:00 | 1m18s789ms925µs944ns | retry proposal 29570825176b0efa: reasonNewLeader | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 224 |
| 2018-06-04 19:48:28.578104+00:00 | 1m18s790ms46µs966ns | applied timestamp cache | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 224 |
| 2018-06-04 19:48:28.57827+00:00 | 1m18s790ms213µs149ns | evaluated request | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 224 |
| 2018-06-04 19:48:28.578289+00:00 | 1m18s790ms231µs822ns | acquired {raft,replica}mu | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 224 |
| 2018-06-04 19:48:28.578901+00:00 | 1m18s790ms844µs385ns | retry proposal 76226963df5b874c: reasonNewLeaderOrConfigChange | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 224 |
| 2018-06-04 19:48:28.578946+00:00 | 1m18s790ms888µs845ns | applied timestamp cache | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 224 |
| 2018-06-04 19:48:28.579037+00:00 | 1m18s790ms980µs628ns | evaluated request | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 224 |
| 2018-06-04 19:48:28.579051+00:00 | 1m18s790ms994µs784ns | acquired {raft,replica}mu | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 224 |
| 2018-06-04 19:48:28.580183+00:00 | 1m18s792ms126µs516ns | command proposed from replica (n1,s1):1 with lease #5 incompatible | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 224 |
| | | to repl=(n1,s1):1 seq=6 start=1528141568.367800593,0 epo=3 | | | | |
| | | pro=1528141704.179807377,0 | | | | |
| 2018-06-04 19:48:28.580209+00:00 | 1m18s792ms152µs278ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 224 |
| | | seq=6 start=1528141568.367800593,0 epo=3 pro=1528141704.179807377,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.367800593,0 epo=3 | | | | |
| | | pro=1528141704.179807377,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:28.580296+00:00 | 1m18s792ms239µs581ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=6 | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | [async] storage.pendingLeaseRequest: requesting lease | 224 |
| | | start=1528141568.367800593,0 epo=3 pro=1528141704.179807377,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.367800593,0 epo=3 | | | | |
| | | pro=1528141704.179807377,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:28.58035+00:00 | 1m18s792ms293µs389ns | read has no clock uncertainty | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | /cockroach.roachpb.Internal/Batch | 222 |
| 2018-06-04 19:48:28.580357+00:00 | 1m18s792ms299µs942ns | command queue | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | /cockroach.roachpb.Internal/Batch | 222 |
| 2018-06-04 19:48:28.580366+00:00 | 1m18s792ms309µs349ns | waiting for read lock | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | /cockroach.roachpb.Internal/Batch | 222 |
| 2018-06-04 19:48:28.580381+00:00 | 1m18s792ms324µs518ns | read completed | [n1,s1,r62/1:/Table/52/1/3{800-900}] | | /cockroach.roachpb.Internal/Batch | 222 |
| 2018-06-04 19:48:28.580523+00:00 | 1m18s792ms466µs315ns | === SPAN START: dist sender === | | | dist sender | 228 |
| 2018-06-04 19:48:28.580545+00:00 | 1m18s792ms487µs855ns | querying next range at /Table/52/1/3900 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 228 |
| 2018-06-04 19:48:28.580578+00:00 | 1m18s792ms521µs592ns | r63: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 228 |
| 2018-06-04 19:48:28.580602+00:00 | 1m18s792ms545µs571ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 228 |
| 2018-06-04 19:48:28.580618+00:00 | 1m18s792ms561µs253ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 229 |
| 2018-06-04 19:48:28.580622+00:00 | 1m18s792ms565µs673ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 229 |
| 2018-06-04 19:48:28.580628+00:00 | 1m18s792ms571µs649ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 229 |
| 2018-06-04 19:48:28.580727+00:00 | 1m18s792ms670µs631ns | read-only path | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | /cockroach.roachpb.Internal/Batch | 229 |
| 2018-06-04 19:48:28.580733+00:00 | 1m18s792ms675µs931ns | request range lease (attempt #1) | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | /cockroach.roachpb.Internal/Batch | 229 |
| 2018-06-04 19:48:28.58074+00:00 | 1m18s792ms683µs452ns | === SPAN START: request range lease === | | | request range lease | 230 |
| 2018-06-04 19:48:28.580747+00:00 | 1m18s792ms690µs472ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 231 |
| | | === | | | | |
| 2018-06-04 19:48:28.580768+00:00 | 1m18s792ms711µs469ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 232 |
| 2018-06-04 19:48:28.580844+00:00 | 1m18s792ms787µs22ns | === SPAN START: dist sender === | | | dist sender | 233 |
| 2018-06-04 19:48:28.580856+00:00 | 1m18s792ms799µs358ns | querying next range at /System/NodeLiveness/1 | [s1,r63/1:/Table/52/1/{3900-4000},txn=a744fba4,n1] | | dist sender | 233 |
| 2018-06-04 19:48:28.58088+00:00 | 1m18s792ms823µs257ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n2,s2):2 | [s1,r63/1:/Table/52/1/{3900-4000},txn=a744fba4,n1] | | dist sender | 233 |
| 2018-06-04 19:48:28.580899+00:00 | 1m18s792ms842µs23ns | sending request to alex-laptop.local:26259 | [s1,r63/1:/Table/52/1/{3900-4000},txn=a744fba4,n1] | | dist sender | 233 |
| 2018-06-04 19:48:28.580919+00:00 | 1m18s792ms862µs172ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 234 |
| 2018-06-04 19:48:28.581297+00:00 | 1m18s793ms239µs947ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 437 |
| 2018-06-04 19:48:28.581339+00:00 | 1m18s793ms282µs349ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n2] | | /cockroach.roachpb.Internal/Batch | 437 |
| 2018-06-04 19:48:28.581348+00:00 | 1m18s793ms291µs514ns | executing 3 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 437 |
| 2018-06-04 19:48:28.581356+00:00 | 1m18s793ms298µs837ns | read-write path | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 437 |
| 2018-06-04 19:48:28.581372+00:00 | 1m18s793ms315µs424ns | command queue | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 437 |
| 2018-06-04 19:48:28.581412+00:00 | 1m18s793ms355µs622ns | applied timestamp cache | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 437 |
| 2018-06-04 19:48:28.581494+00:00 | 1m18s793ms437µs656ns | evaluated request | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 437 |
| 2018-06-04 19:48:28.581499+00:00 | 1m18s793ms442µs128ns | acquired {raft,replica}mu | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 437 |
| 2018-06-04 19:48:28.582163+00:00 | 1m18s794ms106µs510ns | applying command | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 437 |
| 2018-06-04 19:48:28.582496+00:00 | 1m18s794ms439µs530ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 437 |
| 2018-06-04 19:48:28.582524+00:00 | 1m18s794ms467µs499ns | gossiping 1 node liveness record(s) from span | [n2,s2,r3/2:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 437 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:48:28.583001+00:00 | 1m18s794ms944µs292ns | heartbeat 1528141718.080771949,0 | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | liveness heartbeat | 232 |
| 2018-06-04 19:48:28.583011+00:00 | 1m18s794ms954µs686ns | read-write path | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | [async] storage.pendingLeaseRequest: requesting lease | 231 |
| 2018-06-04 19:48:28.583021+00:00 | 1m18s794ms964µs141ns | applied timestamp cache | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | [async] storage.pendingLeaseRequest: requesting lease | 231 |
| 2018-06-04 19:48:28.583114+00:00 | 1m18s795ms56µs863ns | evaluated request | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | [async] storage.pendingLeaseRequest: requesting lease | 231 |
| 2018-06-04 19:48:28.583117+00:00 | 1m18s795ms59µs995ns | acquired {raft,replica}mu | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | [async] storage.pendingLeaseRequest: requesting lease | 231 |
| 2018-06-04 19:48:31.572854+00:00 | 1m21s784ms797µs175ns | retry proposal 0aced7a81d41d02a: reasonTicks | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | [async] storage.pendingLeaseRequest: requesting lease | 231 |
| 2018-06-04 19:48:31.572965+00:00 | 1m21s784ms908µs383ns | applied timestamp cache | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | [async] storage.pendingLeaseRequest: requesting lease | 231 |
| 2018-06-04 19:48:31.573082+00:00 | 1m21s785ms25µs15ns | evaluated request | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | [async] storage.pendingLeaseRequest: requesting lease | 231 |
| 2018-06-04 19:48:31.573085+00:00 | 1m21s785ms27µs968ns | acquired {raft,replica}mu | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | [async] storage.pendingLeaseRequest: requesting lease | 231 |
| 2018-06-04 19:48:32.774857+00:00 | 1m22s986ms799µs896ns | retry proposal 75125e91f1bffc71: reasonNewLeader | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | [async] storage.pendingLeaseRequest: requesting lease | 231 |
| 2018-06-04 19:48:32.774887+00:00 | 1m22s986ms829µs837ns | applied timestamp cache | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | [async] storage.pendingLeaseRequest: requesting lease | 231 |
| 2018-06-04 19:48:32.774968+00:00 | 1m22s986ms911µs53ns | evaluated request | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | [async] storage.pendingLeaseRequest: requesting lease | 231 |
| 2018-06-04 19:48:32.774971+00:00 | 1m22s986ms914µs750ns | acquired {raft,replica}mu | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | [async] storage.pendingLeaseRequest: requesting lease | 231 |
| 2018-06-04 19:48:32.776201+00:00 | 1m22s988ms144µs297ns | retry proposal 6a471c176141d518: reasonNewLeaderOrConfigChange | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | [async] storage.pendingLeaseRequest: requesting lease | 231 |
| 2018-06-04 19:48:32.776235+00:00 | 1m22s988ms178µs390ns | applied timestamp cache | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | [async] storage.pendingLeaseRequest: requesting lease | 231 |
| 2018-06-04 19:48:32.776325+00:00 | 1m22s988ms268µs323ns | evaluated request | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | [async] storage.pendingLeaseRequest: requesting lease | 231 |
| 2018-06-04 19:48:32.776418+00:00 | 1m22s988ms361µs692ns | acquired {raft,replica}mu | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | [async] storage.pendingLeaseRequest: requesting lease | 231 |
| 2018-06-04 19:48:32.778426+00:00 | 1m22s990ms369µs353ns | command proposed from replica (n1,s1):1 with lease #3 incompatible | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | [async] storage.pendingLeaseRequest: requesting lease | 231 |
| | | to repl=(n3,s3):2 seq=4 start=1528141709.302721661,0 epo=2 | | | | |
| | | pro=1528141709.302724098,0 | | | | |
| 2018-06-04 19:48:32.778451+00:00 | 1m22s990ms394µs638ns | applying command with forced error: cannot replace lease repl=(n3,s3):2 | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | [async] storage.pendingLeaseRequest: requesting lease | 231 |
| | | seq=4 start=1528141709.302721661,0 epo=2 pro=1528141709.302724098,0 | | | | |
| | | with repl=(n1,s1):1 seq=4 start=1528141568.367917517,0 epo=3 | | | | |
| | | pro=1528141708.580735302,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:32.778539+00:00 | 1m22s990ms482µs716ns | replica.Send got error: cannot replace lease repl=(n3,s3):2 seq=4 | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | [async] storage.pendingLeaseRequest: requesting lease | 231 |
| | | start=1528141709.302721661,0 epo=2 pro=1528141709.302724098,0 | | | | |
| | | with repl=(n1,s1):1 seq=4 start=1528141568.367917517,0 epo=3 | | | | |
| | | pro=1528141708.580735302,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:32.778621+00:00 | 1m22s990ms563µs993ns | replica.Send got error: [NotLeaseHolderError] r63: replica | [n1,s1,r63/1:/Table/52/1/{3900-4000}] | | /cockroach.roachpb.Internal/Batch | 229 |
| | | (n1,s1):1 not lease holder; current lease is repl=(n3,s3):2 seq=4 | | | | |
| | | start=1528141709.302721661,0 epo=2 pro=1528141709.302724098,0 | | | | |
| 2018-06-04 19:48:32.778636+00:00 | 1m22s990ms579µs457ns | *roachpb.NotLeaseHolderError | [n1] | | /cockroach.roachpb.Internal/Batch | 229 |
| 2018-06-04 19:48:32.778677+00:00 | 1m22s990ms620µs417ns | application error: [NotLeaseHolderError] r63: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 228 |
| | | not lease holder; current lease is repl=(n3,s3):2 seq=4 | | | | |
| | | start=1528141709.302721661,0 epo=2 pro=1528141709.302724098,0 | | | | |
| 2018-06-04 19:48:32.77869+00:00 | 1m22s990ms633µs163ns | error: {(err: [NotLeaseHolderError] r63: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 228 |
| | | not lease holder; current lease is repl=(n3,s3):2 seq=4 | | | | |
| | | start=1528141709.302721661,0 epo=2 pro=1528141709.302724098,0) <nil>}; | | | | |
| | | trying next peer (n3,s3):2 | | | | |
| 2018-06-04 19:48:32.778714+00:00 | 1m22s990ms657µs286ns | sending request to alex-laptop.local:26261 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 228 |
| 2018-06-04 19:48:32.778739+00:00 | 1m22s990ms682µs427ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 235 |
| 2018-06-04 19:48:32.779063+00:00 | 1m22s991ms6µs442ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 438 |
| 2018-06-04 19:48:32.779097+00:00 | 1m22s991ms39µs937ns | 1 LeaseInfo | [n3] | | /cockroach.roachpb.Internal/Batch | 438 |
| 2018-06-04 19:48:32.779104+00:00 | 1m22s991ms47µs68ns | executing 1 requests | [n3,s3] | | /cockroach.roachpb.Internal/Batch | 438 |
| 2018-06-04 19:48:32.779111+00:00 | 1m22s991ms54µs55ns | read-only path | [n3,s3,r63/2:/Table/52/1/{3900-4000}] | | /cockroach.roachpb.Internal/Batch | 438 |
| 2018-06-04 19:48:32.779135+00:00 | 1m22s991ms78µs577ns | command queue | [n3,s3,r63/2:/Table/52/1/{3900-4000}] | | /cockroach.roachpb.Internal/Batch | 438 |
| 2018-06-04 19:48:32.779141+00:00 | 1m22s991ms84µs609ns | waiting for read lock | [n3,s3,r63/2:/Table/52/1/{3900-4000}] | | /cockroach.roachpb.Internal/Batch | 438 |
| 2018-06-04 19:48:32.779156+00:00 | 1m22s991ms99µs530ns | read completed | [n3,s3,r63/2:/Table/52/1/{3900-4000}] | | /cockroach.roachpb.Internal/Batch | 438 |
| 2018-06-04 19:48:32.779598+00:00 | 1m22s991ms541µs675ns | === SPAN START: dist sender === | | | dist sender | 236 |
| 2018-06-04 19:48:32.77963+00:00 | 1m22s991ms573µs151ns | querying next range at /Table/52/1/4000 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 236 |
| 2018-06-04 19:48:32.779692+00:00 | 1m22s991ms634µs953ns | r64: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 236 |
| 2018-06-04 19:48:32.779711+00:00 | 1m22s991ms653µs967ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 236 |
| 2018-06-04 19:48:32.779727+00:00 | 1m22s991ms670µs581ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 237 |
| 2018-06-04 19:48:32.779733+00:00 | 1m22s991ms676µs216ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 237 |
| 2018-06-04 19:48:32.779739+00:00 | 1m22s991ms682µs512ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 237 |
| 2018-06-04 19:48:32.779746+00:00 | 1m22s991ms689µs759ns | read-only path | [n1,s1,r64/1:/Table/52/1/4{000-100}] | | /cockroach.roachpb.Internal/Batch | 237 |
| 2018-06-04 19:48:32.77977+00:00 | 1m22s991ms712µs956ns | replica.Send got error: [NotLeaseHolderError] r64: replica | [n1,s1,r64/1:/Table/52/1/4{000-100}] | | /cockroach.roachpb.Internal/Batch | 237 |
| | | (n1,s1):1 not lease holder; current lease is repl=(n2,s2):3 seq=6 | | | | |
| | | start=1528141709.300520764,0 epo=2 pro=1528141709.300522636,0 | | | | |
| 2018-06-04 19:48:32.779782+00:00 | 1m22s991ms725µs489ns | *roachpb.NotLeaseHolderError | [n1] | | /cockroach.roachpb.Internal/Batch | 237 |
| 2018-06-04 19:48:32.779813+00:00 | 1m22s991ms756µs465ns | application error: [NotLeaseHolderError] r64: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 236 |
| | | not lease holder; current lease is repl=(n2,s2):3 seq=6 | | | | |
| | | start=1528141709.300520764,0 epo=2 pro=1528141709.300522636,0 | | | | |
| 2018-06-04 19:48:32.779826+00:00 | 1m22s991ms769µs696ns | error: {(err: [NotLeaseHolderError] r64: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 236 |
| | | not lease holder; current lease is repl=(n2,s2):3 seq=6 | | | | |
| | | start=1528141709.300520764,0 epo=2 pro=1528141709.300522636,0) <nil>}; | | | | |
| | | trying next peer (n2,s2):3 | | | | |
| 2018-06-04 19:48:32.779856+00:00 | 1m22s991ms799µs774ns | sending request to alex-laptop.local:26259 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 236 |
| 2018-06-04 19:48:32.7799+00:00 | 1m22s991ms843µs537ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 238 |
| 2018-06-04 19:48:32.780364+00:00 | 1m22s992ms307µs391ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 439 |
| 2018-06-04 19:48:32.7804+00:00 | 1m22s992ms342µs973ns | 1 LeaseInfo | [n2] | | /cockroach.roachpb.Internal/Batch | 439 |
| 2018-06-04 19:48:32.780407+00:00 | 1m22s992ms350µs565ns | executing 1 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 439 |
| 2018-06-04 19:48:32.780415+00:00 | 1m22s992ms358µs560ns | read-only path | [n2,s2,r64/3:/Table/52/1/4{000-100}] | | /cockroach.roachpb.Internal/Batch | 439 |
| 2018-06-04 19:48:32.780425+00:00 | 1m22s992ms368µs160ns | command queue | [n2,s2,r64/3:/Table/52/1/4{000-100}] | | /cockroach.roachpb.Internal/Batch | 439 |
| 2018-06-04 19:48:32.780432+00:00 | 1m22s992ms374µs967ns | waiting for read lock | [n2,s2,r64/3:/Table/52/1/4{000-100}] | | /cockroach.roachpb.Internal/Batch | 439 |
| 2018-06-04 19:48:32.78046+00:00 | 1m22s992ms402µs992ns | read completed | [n2,s2,r64/3:/Table/52/1/4{000-100}] | | /cockroach.roachpb.Internal/Batch | 439 |
| 2018-06-04 19:48:32.780884+00:00 | 1m22s992ms827µs401ns | === SPAN START: dist sender === | | | dist sender | 239 |
| 2018-06-04 19:48:32.780902+00:00 | 1m22s992ms845µs749ns | querying next range at /Table/52/1/4100 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 239 |
| 2018-06-04 19:48:32.780934+00:00 | 1m22s992ms877µs622ns | r65: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 239 |
| 2018-06-04 19:48:32.780952+00:00 | 1m22s992ms895µs632ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 239 |
| 2018-06-04 19:48:32.780969+00:00 | 1m22s992ms912µs676ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 240 |
| 2018-06-04 19:48:32.780975+00:00 | 1m22s992ms918µs14ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 240 |
| 2018-06-04 19:48:32.780982+00:00 | 1m22s992ms924µs821ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 240 |
| 2018-06-04 19:48:32.780989+00:00 | 1m22s992ms932µs468ns | read-only path | [n1,s1,r65/1:/Table/52/1/4{100-200}] | | /cockroach.roachpb.Internal/Batch | 240 |
| 2018-06-04 19:48:32.780995+00:00 | 1m22s992ms938µs86ns | read has no clock uncertainty | [n1,s1,r65/1:/Table/52/1/4{100-200}] | | /cockroach.roachpb.Internal/Batch | 240 |
| 2018-06-04 19:48:32.780999+00:00 | 1m22s992ms942µs246ns | command queue | [n1,s1,r65/1:/Table/52/1/4{100-200}] | | /cockroach.roachpb.Internal/Batch | 240 |
| 2018-06-04 19:48:32.781005+00:00 | 1m22s992ms948µs377ns | waiting for read lock | [n1,s1,r65/1:/Table/52/1/4{100-200}] | | /cockroach.roachpb.Internal/Batch | 240 |
| 2018-06-04 19:48:32.781017+00:00 | 1m22s992ms959µs834ns | read completed | [n1,s1,r65/1:/Table/52/1/4{100-200}] | | /cockroach.roachpb.Internal/Batch | 240 |
| 2018-06-04 19:48:32.781078+00:00 | 1m22s993ms21µs104ns | === SPAN START: dist sender === | | | dist sender | 241 |
| 2018-06-04 19:48:32.781163+00:00 | 1m22s993ms105µs877ns | querying next range at /Table/52/1/4200 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 241 |
| 2018-06-04 19:48:32.78119+00:00 | 1m22s993ms133µs269ns | r66: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 241 |
| 2018-06-04 19:48:32.781206+00:00 | 1m22s993ms149µs525ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 241 |
| 2018-06-04 19:48:32.781238+00:00 | 1m22s993ms181µs246ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 242 |
| 2018-06-04 19:48:32.781244+00:00 | 1m22s993ms187µs341ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 242 |
| 2018-06-04 19:48:32.78125+00:00 | 1m22s993ms193µs308ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 242 |
| 2018-06-04 19:48:32.781257+00:00 | 1m22s993ms199µs969ns | read-only path | [n1,s1,r66/1:/Table/52/1/4{200-300}] | | /cockroach.roachpb.Internal/Batch | 242 |
| 2018-06-04 19:48:32.781263+00:00 | 1m22s993ms205µs915ns | request range lease (attempt #1) | [n1,s1,r66/1:/Table/52/1/4{200-300}] | | /cockroach.roachpb.Internal/Batch | 242 |
| 2018-06-04 19:48:32.781272+00:00 | 1m22s993ms214µs839ns | === SPAN START: request range lease === | | | request range lease | 243 |
| 2018-06-04 19:48:32.781285+00:00 | 1m22s993ms228µs309ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 244 |
| | | === | | | | |
| 2018-06-04 19:48:32.781324+00:00 | 1m22s993ms267µs577ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 245 |
| 2018-06-04 19:48:32.781471+00:00 | 1m22s993ms414µs400ns | === SPAN START: dist sender === | | | dist sender | 246 |
| 2018-06-04 19:48:32.781492+00:00 | 1m22s993ms435µs505ns | querying next range at /System/NodeLiveness/1 | [s1,r66/1:/Table/52/1/4{200-300},txn=77460716,n1] | | dist sender | 246 |
| 2018-06-04 19:48:32.781523+00:00 | 1m22s993ms465µs832ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n3,s3):3 | [s1,r66/1:/Table/52/1/4{200-300},txn=77460716,n1] | | dist sender | 246 |
| 2018-06-04 19:48:32.781559+00:00 | 1m22s993ms501µs895ns | sending request to alex-laptop.local:26261 | [s1,r66/1:/Table/52/1/4{200-300},txn=77460716,n1] | | dist sender | 246 |
| 2018-06-04 19:48:32.781578+00:00 | 1m22s993ms521µs532ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 247 |
| 2018-06-04 19:48:32.781984+00:00 | 1m22s993ms927µs642ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 440 |
| 2018-06-04 19:48:32.782016+00:00 | 1m22s993ms959µs674ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n3] | | /cockroach.roachpb.Internal/Batch | 440 |
| 2018-06-04 19:48:32.782024+00:00 | 1m22s993ms967µs736ns | executing 3 requests | [n3,s3] | | /cockroach.roachpb.Internal/Batch | 440 |
| 2018-06-04 19:48:32.782032+00:00 | 1m22s993ms975µs344ns | read-write path | [n3,s3,r3/3:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 440 |
| 2018-06-04 19:48:32.782044+00:00 | 1m22s993ms987µs689ns | command queue | [n3,s3,r3/3:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 440 |
| 2018-06-04 19:48:32.782079+00:00 | 1m22s994ms22µs125ns | applied timestamp cache | [n3,s3,r3/3:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 440 |
| 2018-06-04 19:48:32.782195+00:00 | 1m22s994ms138µs86ns | evaluated request | [n3,s3,r3/3:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 440 |
| 2018-06-04 19:48:32.782201+00:00 | 1m22s994ms144µs404ns | acquired {raft,replica}mu | [n3,s3,r3/3:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 440 |
| 2018-06-04 19:48:32.783017+00:00 | 1m22s994ms960µs198ns | applying command | [n3,s3,r3/3:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 440 |
| 2018-06-04 19:48:32.783743+00:00 | 1m22s995ms685µs888ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n3,s3,r3/3:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 440 |
| 2018-06-04 19:48:32.783783+00:00 | 1m22s995ms726µs374ns | gossiping 1 node liveness record(s) from span | [n3,s3,r3/3:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 440 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:48:32.784251+00:00 | 1m22s996ms194µs613ns | heartbeat 1528141722.281330382,0 | [n1,s1,r66/1:/Table/52/1/4{200-300}] | | liveness heartbeat | 245 |
| 2018-06-04 19:48:32.784263+00:00 | 1m22s996ms205µs844ns | read-write path | [n1,s1,r66/1:/Table/52/1/4{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 244 |
| 2018-06-04 19:48:32.78427+00:00 | 1m22s996ms212µs956ns | applied timestamp cache | [n1,s1,r66/1:/Table/52/1/4{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 244 |
| 2018-06-04 19:48:32.784377+00:00 | 1m22s996ms320µs302ns | evaluated request | [n1,s1,r66/1:/Table/52/1/4{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 244 |
| 2018-06-04 19:48:32.78438+00:00 | 1m22s996ms323µs608ns | acquired {raft,replica}mu | [n1,s1,r66/1:/Table/52/1/4{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 244 |
| 2018-06-04 19:48:32.78579+00:00 | 1m22s997ms733µs25ns | command proposed from replica (n1,s1):1 with lease #5 incompatible | [n1,s1,r66/1:/Table/52/1/4{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 244 |
| | | to repl=(n3,s3):2 seq=6 start=1528141712.778205832,0 epo=2 | | | | |
| | | pro=1528141712.778209059,0 | | | | |
| 2018-06-04 19:48:32.785812+00:00 | 1m22s997ms755µs212ns | applying command with forced error: cannot replace lease repl=(n3,s3):2 | [n1,s1,r66/1:/Table/52/1/4{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 244 |
| | | seq=6 start=1528141712.778205832,0 epo=2 pro=1528141712.778209059,0 | | | | |
| | | with repl=(n1,s1):1 seq=7 start=1528141712.781259755,0 epo=3 | | | | |
| | | pro=1528141712.781265746,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:32.786022+00:00 | 1m22s997ms965µs672ns | replica.Send got error: cannot replace lease repl=(n3,s3):2 seq=6 | [n1,s1,r66/1:/Table/52/1/4{200-300}] | | [async] storage.pendingLeaseRequest: requesting lease | 244 |
| | | start=1528141712.778205832,0 epo=2 pro=1528141712.778209059,0 | | | | |
| | | with repl=(n1,s1):1 seq=7 start=1528141712.781259755,0 epo=3 | | | | |
| | | pro=1528141712.781265746,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:32.786073+00:00 | 1m22s998ms16µs506ns | replica.Send got error: [NotLeaseHolderError] r66: replica | [n1,s1,r66/1:/Table/52/1/4{200-300}] | | /cockroach.roachpb.Internal/Batch | 242 |
| | | (n1,s1):1 not lease holder; current lease is repl=(n3,s3):2 seq=6 | | | | |
| | | start=1528141712.778205832,0 epo=2 pro=1528141712.778209059,0 | | | | |
| 2018-06-04 19:48:32.786087+00:00 | 1m22s998ms30µs435ns | *roachpb.NotLeaseHolderError | [n1] | | /cockroach.roachpb.Internal/Batch | 242 |
| 2018-06-04 19:48:32.786105+00:00 | 1m22s998ms47µs965ns | application error: [NotLeaseHolderError] r66: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 241 |
| | | not lease holder; current lease is repl=(n3,s3):2 seq=6 | | | | |
| | | start=1528141712.778205832,0 epo=2 pro=1528141712.778209059,0 | | | | |
| 2018-06-04 19:48:32.786116+00:00 | 1m22s998ms58µs880ns | error: {(err: [NotLeaseHolderError] r66: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 241 |
| | | not lease holder; current lease is repl=(n3,s3):2 seq=6 | | | | |
| | | start=1528141712.778205832,0 epo=2 pro=1528141712.778209059,0) <nil>}; | | | | |
| | | trying next peer (n3,s3):2 | | | | |
| 2018-06-04 19:48:32.786135+00:00 | 1m22s998ms78µs135ns | sending request to alex-laptop.local:26261 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 241 |
| 2018-06-04 19:48:32.786159+00:00 | 1m22s998ms102µs205ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 248 |
| 2018-06-04 19:48:32.786622+00:00 | 1m22s998ms565µs644ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 441 |
| 2018-06-04 19:48:32.786671+00:00 | 1m22s998ms614µs413ns | 1 LeaseInfo | [n3] | | /cockroach.roachpb.Internal/Batch | 441 |
| 2018-06-04 19:48:32.786679+00:00 | 1m22s998ms622µs407ns | executing 1 requests | [n3,s3] | | /cockroach.roachpb.Internal/Batch | 441 |
| 2018-06-04 19:48:32.786686+00:00 | 1m22s998ms628µs865ns | read-only path | [n3,s3,r66/2:/Table/52/1/4{200-300}] | | /cockroach.roachpb.Internal/Batch | 441 |
| 2018-06-04 19:48:32.786692+00:00 | 1m22s998ms635µs470ns | command queue | [n3,s3,r66/2:/Table/52/1/4{200-300}] | | /cockroach.roachpb.Internal/Batch | 441 |
| 2018-06-04 19:48:32.786696+00:00 | 1m22s998ms639µs806ns | waiting for read lock | [n3,s3,r66/2:/Table/52/1/4{200-300}] | | /cockroach.roachpb.Internal/Batch | 441 |
| 2018-06-04 19:48:32.786759+00:00 | 1m22s998ms702µs377ns | read completed | [n3,s3,r66/2:/Table/52/1/4{200-300}] | | /cockroach.roachpb.Internal/Batch | 441 |
| 2018-06-04 19:48:32.787073+00:00 | 1m22s999ms16µs21ns | === SPAN START: dist sender === | | | dist sender | 249 |
| 2018-06-04 19:48:32.787092+00:00 | 1m22s999ms35µs733ns | querying next range at /Table/52/1/4300 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 249 |
| 2018-06-04 19:48:32.787122+00:00 | 1m22s999ms64µs857ns | r67: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 249 |
| 2018-06-04 19:48:32.787137+00:00 | 1m22s999ms80µs565ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 249 |
| 2018-06-04 19:48:32.787182+00:00 | 1m22s999ms125µs258ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 250 |
| 2018-06-04 19:48:32.787188+00:00 | 1m22s999ms131µs11ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 250 |
| 2018-06-04 19:48:32.787194+00:00 | 1m22s999ms137µs528ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 250 |
| 2018-06-04 19:48:32.7872+00:00 | 1m22s999ms143µs98ns | read-only path | [n1,s1,r67/1:/Table/52/1/4{300-400}] | | /cockroach.roachpb.Internal/Batch | 250 |
| 2018-06-04 19:48:32.78722+00:00 | 1m22s999ms163µs350ns | replica.Send got error: [NotLeaseHolderError] r67: replica | [n1,s1,r67/1:/Table/52/1/4{300-400}] | | /cockroach.roachpb.Internal/Batch | 250 |
| | | (n1,s1):1 not lease holder; current lease is repl=(n2,s2):3 seq=4 | | | | |
| | | start=1528141709.303521970,0 epo=2 pro=1528141709.303525049,0 | | | | |
| 2018-06-04 19:48:32.787234+00:00 | 1m22s999ms176µs816ns | *roachpb.NotLeaseHolderError | [n1] | | /cockroach.roachpb.Internal/Batch | 250 |
| 2018-06-04 19:48:32.787247+00:00 | 1m22s999ms189µs963ns | application error: [NotLeaseHolderError] r67: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 249 |
| | | not lease holder; current lease is repl=(n2,s2):3 seq=4 | | | | |
| | | start=1528141709.303521970,0 epo=2 pro=1528141709.303525049,0 | | | | |
| 2018-06-04 19:48:32.787257+00:00 | 1m22s999ms200µs267ns | error: {(err: [NotLeaseHolderError] r67: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 249 |
| | | not lease holder; current lease is repl=(n2,s2):3 seq=4 | | | | |
| | | start=1528141709.303521970,0 epo=2 pro=1528141709.303525049,0) <nil>}; | | | | |
| | | trying next peer (n2,s2):3 | | | | |
| 2018-06-04 19:48:32.78727+00:00 | 1m22s999ms213µs727ns | sending request to alex-laptop.local:26259 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 249 |
| 2018-06-04 19:48:32.78729+00:00 | 1m22s999ms233µs272ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 251 |
| 2018-06-04 19:48:32.787604+00:00 | 1m22s999ms547µs568ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 442 |
| 2018-06-04 19:48:32.787631+00:00 | 1m22s999ms573µs859ns | 1 LeaseInfo | [n2] | | /cockroach.roachpb.Internal/Batch | 442 |
| 2018-06-04 19:48:32.787636+00:00 | 1m22s999ms579µs395ns | executing 1 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 442 |
| 2018-06-04 19:48:32.787644+00:00 | 1m22s999ms586µs978ns | read-only path | [n2,s2,r67/3:/Table/52/1/4{300-400}] | | /cockroach.roachpb.Internal/Batch | 442 |
| 2018-06-04 19:48:32.787653+00:00 | 1m22s999ms595µs834ns | command queue | [n2,s2,r67/3:/Table/52/1/4{300-400}] | | /cockroach.roachpb.Internal/Batch | 442 |
| 2018-06-04 19:48:32.787657+00:00 | 1m22s999ms600µs245ns | waiting for read lock | [n2,s2,r67/3:/Table/52/1/4{300-400}] | | /cockroach.roachpb.Internal/Batch | 442 |
| 2018-06-04 19:48:32.787667+00:00 | 1m22s999ms610µs270ns | read completed | [n2,s2,r67/3:/Table/52/1/4{300-400}] | | /cockroach.roachpb.Internal/Batch | 442 |
| 2018-06-04 19:48:32.78799+00:00 | 1m22s999ms933µs100ns | === SPAN START: dist sender === | | | dist sender | 252 |
| 2018-06-04 19:48:32.788006+00:00 | 1m22s999ms949µs693ns | querying next range at /Table/52/1/4400 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 252 |
| 2018-06-04 19:48:32.788046+00:00 | 1m22s999ms989µs72ns | r68: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 252 |
| 2018-06-04 19:48:32.788075+00:00 | 1m23s18µs777ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 252 |
| 2018-06-04 19:48:32.788101+00:00 | 1m23s44µs221ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 253 |
| 2018-06-04 19:48:32.788107+00:00 | 1m23s49µs959ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 253 |
| 2018-06-04 19:48:32.788113+00:00 | 1m23s55µs952ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 253 |
| 2018-06-04 19:48:32.788118+00:00 | 1m23s61µs687ns | read-only path | [n1,s1,r68/1:/Table/52/1/4{400-500}] | | /cockroach.roachpb.Internal/Batch | 253 |
| 2018-06-04 19:48:32.788127+00:00 | 1m23s69µs976ns | request range lease (attempt #1) | [n1,s1,r68/1:/Table/52/1/4{400-500}] | | /cockroach.roachpb.Internal/Batch | 253 |
| 2018-06-04 19:48:35.776205+00:00 | 1m25s988ms148µs586ns | read has no clock uncertainty | [n1,s1,r68/1:/Table/52/1/4{400-500}] | | /cockroach.roachpb.Internal/Batch | 253 |
| 2018-06-04 19:48:35.776213+00:00 | 1m25s988ms156µs96ns | command queue | [n1,s1,r68/1:/Table/52/1/4{400-500}] | | /cockroach.roachpb.Internal/Batch | 253 |
| 2018-06-04 19:48:35.776224+00:00 | 1m25s988ms167µs770ns | waiting for read lock | [n1,s1,r68/1:/Table/52/1/4{400-500}] | | /cockroach.roachpb.Internal/Batch | 253 |
| 2018-06-04 19:48:35.77624+00:00 | 1m25s988ms183µs432ns | read completed | [n1,s1,r68/1:/Table/52/1/4{400-500}] | | /cockroach.roachpb.Internal/Batch | 253 |
| 2018-06-04 19:48:35.776419+00:00 | 1m25s988ms362µs299ns | === SPAN START: dist sender === | | | dist sender | 254 |
| 2018-06-04 19:48:35.776442+00:00 | 1m25s988ms385µs221ns | querying next range at /Table/52/1/4500 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 254 |
| 2018-06-04 19:48:35.776507+00:00 | 1m25s988ms450µs632ns | r69: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 254 |
| 2018-06-04 19:48:35.776541+00:00 | 1m25s988ms483µs915ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 254 |
| 2018-06-04 19:48:35.776571+00:00 | 1m25s988ms514µs426ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 255 |
| 2018-06-04 19:48:35.776576+00:00 | 1m25s988ms518µs978ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 255 |
| 2018-06-04 19:48:35.776581+00:00 | 1m25s988ms524µs565ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 255 |
| 2018-06-04 19:48:35.776587+00:00 | 1m25s988ms530µs635ns | read-only path | [n1,s1,r69/1:/Table/52/1/4{500-600}] | | /cockroach.roachpb.Internal/Batch | 255 |
| 2018-06-04 19:48:35.776605+00:00 | 1m25s988ms548µs598ns | replica.Send got error: [NotLeaseHolderError] r69: replica | [n1,s1,r69/1:/Table/52/1/4{500-600}] | | /cockroach.roachpb.Internal/Batch | 255 |
| | | (n1,s1):1 not lease holder; current lease is repl=(n3,s3):2 seq=6 | | | | |
| | | start=1528141712.783689295,0 epo=2 pro=1528141712.783691617,0 | | | | |
| 2018-06-04 19:48:35.776629+00:00 | 1m25s988ms572µs727ns | *roachpb.NotLeaseHolderError | [n1] | | /cockroach.roachpb.Internal/Batch | 255 |
| 2018-06-04 19:48:35.776646+00:00 | 1m25s988ms589µs702ns | application error: [NotLeaseHolderError] r69: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 254 |
| | | not lease holder; current lease is repl=(n3,s3):2 seq=6 | | | | |
| | | start=1528141712.783689295,0 epo=2 pro=1528141712.783691617,0 | | | | |
| 2018-06-04 19:48:35.776655+00:00 | 1m25s988ms598µs359ns | error: {(err: [NotLeaseHolderError] r69: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 254 |
| | | not lease holder; current lease is repl=(n3,s3):2 seq=6 | | | | |
| | | start=1528141712.783689295,0 epo=2 pro=1528141712.783691617,0) <nil>}; | | | | |
| | | trying next peer (n3,s3):2 | | | | |
| 2018-06-04 19:48:35.776664+00:00 | 1m25s988ms607µs451ns | sending request to alex-laptop.local:26261 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 254 |
| 2018-06-04 19:48:35.776685+00:00 | 1m25s988ms628µs601ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 256 |
| 2018-06-04 19:48:35.777003+00:00 | 1m25s988ms946µs327ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 443 |
| 2018-06-04 19:48:35.777053+00:00 | 1m25s988ms996µs551ns | 1 LeaseInfo | [n3] | | /cockroach.roachpb.Internal/Batch | 443 |
| 2018-06-04 19:48:35.777062+00:00 | 1m25s989ms5µs316ns | executing 1 requests | [n3,s3] | | /cockroach.roachpb.Internal/Batch | 443 |
| 2018-06-04 19:48:35.77707+00:00 | 1m25s989ms13µs196ns | read-only path | [n3,s3,r69/2:/Table/52/1/4{500-600}] | | /cockroach.roachpb.Internal/Batch | 443 |
| 2018-06-04 19:48:35.777089+00:00 | 1m25s989ms32µs721ns | command queue | [n3,s3,r69/2:/Table/52/1/4{500-600}] | | /cockroach.roachpb.Internal/Batch | 443 |
| 2018-06-04 19:48:35.777098+00:00 | 1m25s989ms41µs602ns | waiting for read lock | [n3,s3,r69/2:/Table/52/1/4{500-600}] | | /cockroach.roachpb.Internal/Batch | 443 |
| 2018-06-04 19:48:35.777112+00:00 | 1m25s989ms55µs365ns | read completed | [n3,s3,r69/2:/Table/52/1/4{500-600}] | | /cockroach.roachpb.Internal/Batch | 443 |
| 2018-06-04 19:48:35.777386+00:00 | 1m25s989ms329µs418ns | === SPAN START: dist sender === | | | dist sender | 257 |
| 2018-06-04 19:48:35.7774+00:00 | 1m25s989ms343µs87ns | querying next range at /Table/52/1/4600 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 257 |
| 2018-06-04 19:48:35.777423+00:00 | 1m25s989ms366µs353ns | r70: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 257 |
| 2018-06-04 19:48:35.777453+00:00 | 1m25s989ms396µs336ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 257 |
| 2018-06-04 19:48:35.777473+00:00 | 1m25s989ms416µs464ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 258 |
| 2018-06-04 19:48:35.777477+00:00 | 1m25s989ms420µs39ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 258 |
| 2018-06-04 19:48:35.777489+00:00 | 1m25s989ms432µs397ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 258 |
| 2018-06-04 19:48:35.777494+00:00 | 1m25s989ms437µs621ns | read-only path | [n1,s1,r70/1:/Table/52/1/4{600-700}] | | /cockroach.roachpb.Internal/Batch | 258 |
| 2018-06-04 19:48:35.777518+00:00 | 1m25s989ms460µs938ns | replica.Send got error: [NotLeaseHolderError] r70: replica | [n1,s1,r70/1:/Table/52/1/4{600-700}] | | /cockroach.roachpb.Internal/Batch | 258 |
| | | (n1,s1):1 not lease holder; current lease is repl=(n2,s2):3 seq=6 | | | | |
| | | start=1528141712.777878890,0 epo=2 pro=1528141712.777881902,0 | | | | |
| 2018-06-04 19:48:35.777525+00:00 | 1m25s989ms468µs626ns | *roachpb.NotLeaseHolderError | [n1] | | /cockroach.roachpb.Internal/Batch | 258 |
| 2018-06-04 19:48:35.777538+00:00 | 1m25s989ms481µs472ns | application error: [NotLeaseHolderError] r70: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 257 |
| | | not lease holder; current lease is repl=(n2,s2):3 seq=6 | | | | |
| | | start=1528141712.777878890,0 epo=2 pro=1528141712.777881902,0 | | | | |
| 2018-06-04 19:48:35.777545+00:00 | 1m25s989ms488µs69ns | error: {(err: [NotLeaseHolderError] r70: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 257 |
| | | not lease holder; current lease is repl=(n2,s2):3 seq=6 | | | | |
| | | start=1528141712.777878890,0 epo=2 pro=1528141712.777881902,0) <nil>}; | | | | |
| | | trying next peer (n2,s2):3 | | | | |
| 2018-06-04 19:48:35.777557+00:00 | 1m25s989ms499µs978ns | sending request to alex-laptop.local:26259 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 257 |
| 2018-06-04 19:48:35.777575+00:00 | 1m25s989ms518µs240ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 259 |
| 2018-06-04 19:48:35.777768+00:00 | 1m25s989ms710µs972ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 444 |
| 2018-06-04 19:48:35.777792+00:00 | 1m25s989ms735µs43ns | 1 LeaseInfo | [n2] | | /cockroach.roachpb.Internal/Batch | 444 |
| 2018-06-04 19:48:35.777798+00:00 | 1m25s989ms741µs240ns | executing 1 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 444 |
| 2018-06-04 19:48:35.777804+00:00 | 1m25s989ms747µs413ns | read-only path | [n2,s2,r70/3:/Table/52/1/4{600-700}] | | /cockroach.roachpb.Internal/Batch | 444 |
| 2018-06-04 19:48:35.777818+00:00 | 1m25s989ms761µs119ns | command queue | [n2,s2,r70/3:/Table/52/1/4{600-700}] | | /cockroach.roachpb.Internal/Batch | 444 |
| 2018-06-04 19:48:35.777822+00:00 | 1m25s989ms765µs362ns | waiting for read lock | [n2,s2,r70/3:/Table/52/1/4{600-700}] | | /cockroach.roachpb.Internal/Batch | 444 |
| 2018-06-04 19:48:35.777831+00:00 | 1m25s989ms774µs718ns | read completed | [n2,s2,r70/3:/Table/52/1/4{600-700}] | | /cockroach.roachpb.Internal/Batch | 444 |
| 2018-06-04 19:48:35.778087+00:00 | 1m25s990ms29µs970ns | === SPAN START: dist sender === | | | dist sender | 260 |
| 2018-06-04 19:48:35.778107+00:00 | 1m25s990ms49µs937ns | querying next range at /Table/52/1/4700 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 260 |
| 2018-06-04 19:48:35.778137+00:00 | 1m25s990ms80µs190ns | r71: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 260 |
| 2018-06-04 19:48:35.778156+00:00 | 1m25s990ms99µs387ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 260 |
| 2018-06-04 19:48:35.778218+00:00 | 1m25s990ms161µs68ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 261 |
| 2018-06-04 19:48:35.778222+00:00 | 1m25s990ms165µs173ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 261 |
| 2018-06-04 19:48:35.778227+00:00 | 1m25s990ms169µs931ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 261 |
| 2018-06-04 19:48:35.778348+00:00 | 1m25s990ms291µs763ns | read-only path | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | /cockroach.roachpb.Internal/Batch | 261 |
| 2018-06-04 19:48:35.778353+00:00 | 1m25s990ms296µs438ns | request range lease (attempt #1) | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | /cockroach.roachpb.Internal/Batch | 261 |
| 2018-06-04 19:48:35.778359+00:00 | 1m25s990ms302µs720ns | === SPAN START: request range lease === | | | request range lease | 262 |
| 2018-06-04 19:48:35.778366+00:00 | 1m25s990ms309µs759ns | === SPAN START: [async] storage.pendingLeaseRequest: requesting lease | | | [async] storage.pendingLeaseRequest: requesting lease | 263 |
| | | === | | | | |
| 2018-06-04 19:48:35.778385+00:00 | 1m25s990ms328µs63ns | === SPAN START: liveness heartbeat === | | | liveness heartbeat | 264 |
| 2018-06-04 19:48:35.778447+00:00 | 1m25s990ms389µs937ns | === SPAN START: dist sender === | | | dist sender | 265 |
| 2018-06-04 19:48:35.778457+00:00 | 1m25s990ms400µs712ns | querying next range at /System/NodeLiveness/1 | [s1,r71/1:/Table/52/1/4{700-800},txn=4cd7e55d,n1] | | dist sender | 265 |
| 2018-06-04 19:48:35.778486+00:00 | 1m25s990ms429µs383ns | r3: sending batch 1 CPut, 1 BeginTxn, 1 EndTxn to (n3,s3):3 | [s1,r71/1:/Table/52/1/4{700-800},txn=4cd7e55d,n1] | | dist sender | 265 |
| 2018-06-04 19:48:35.778506+00:00 | 1m25s990ms449µs496ns | sending request to alex-laptop.local:26261 | [s1,r71/1:/Table/52/1/4{700-800},txn=4cd7e55d,n1] | | dist sender | 265 |
| 2018-06-04 19:48:35.778518+00:00 | 1m25s990ms461µs383ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 266 |
| 2018-06-04 19:48:35.778726+00:00 | 1m25s990ms669µs350ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 445 |
| 2018-06-04 19:48:35.778746+00:00 | 1m25s990ms689µs642ns | 1 CPut, 1 BeginTxn, 1 EndTxn | [n3] | | /cockroach.roachpb.Internal/Batch | 445 |
| 2018-06-04 19:48:35.778752+00:00 | 1m25s990ms695µs516ns | executing 3 requests | [n3,s3] | | /cockroach.roachpb.Internal/Batch | 445 |
| 2018-06-04 19:48:35.778758+00:00 | 1m25s990ms701µs235ns | read-write path | [n3,s3,r3/3:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 445 |
| 2018-06-04 19:48:35.778768+00:00 | 1m25s990ms710µs863ns | command queue | [n3,s3,r3/3:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 445 |
| 2018-06-04 19:48:35.778794+00:00 | 1m25s990ms737µs243ns | applied timestamp cache | [n3,s3,r3/3:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 445 |
| 2018-06-04 19:48:35.778906+00:00 | 1m25s990ms849µs470ns | evaluated request | [n3,s3,r3/3:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 445 |
| 2018-06-04 19:48:35.778909+00:00 | 1m25s990ms852µs739ns | acquired {raft,replica}mu | [n3,s3,r3/3:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 445 |
| 2018-06-04 19:48:35.77947+00:00 | 1m25s991ms413µs655ns | applying command | [n3,s3,r3/3:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 445 |
| 2018-06-04 19:48:35.779742+00:00 | 1m25s991ms685µs352ns | engine stats: {InternalDeleteSkippedCount:0 TimeBoundNumSSTs:0} | [n3,s3,r3/3:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 445 |
| 2018-06-04 19:48:35.779762+00:00 | 1m25s991ms704µs828ns | gossiping 1 node liveness record(s) from span | [n3,s3,r3/3:/System/NodeLiveness{-Max}] | | /cockroach.roachpb.Internal/Batch | 445 |
| | | /System/NodeLiveness/1{-/NULL} | | | | |
| 2018-06-04 19:48:35.780107+00:00 | 1m25s992ms50µs486ns | heartbeat 1528141725.278387864,0 | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | liveness heartbeat | 264 |
| 2018-06-04 19:48:35.780114+00:00 | 1m25s992ms57µs11ns | read-write path | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 263 |
| 2018-06-04 19:48:35.780119+00:00 | 1m25s992ms62µs292ns | applied timestamp cache | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 263 |
| 2018-06-04 19:48:35.780215+00:00 | 1m25s992ms158µs212ns | evaluated request | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 263 |
| 2018-06-04 19:48:35.780218+00:00 | 1m25s992ms161µs229ns | acquired {raft,replica}mu | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 263 |
| 2018-06-04 19:48:38.773623+00:00 | 1m28s985ms566µs40ns | retry proposal 39f3e964a996535b: reasonTicks | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 263 |
| 2018-06-04 19:48:38.773676+00:00 | 1m28s985ms618µs956ns | applied timestamp cache | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 263 |
| 2018-06-04 19:48:38.773855+00:00 | 1m28s985ms798µs497ns | evaluated request | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 263 |
| 2018-06-04 19:48:38.77386+00:00 | 1m28s985ms802µs919ns | acquired {raft,replica}mu | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 263 |
| 2018-06-04 19:48:39.174293+00:00 | 1m29s386ms236µs36ns | retry proposal 1ec080f28b1da913: reasonNewLeader | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 263 |
| 2018-06-04 19:48:39.17435+00:00 | 1m29s386ms293µs528ns | applied timestamp cache | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 263 |
| 2018-06-04 19:48:39.174526+00:00 | 1m29s386ms468µs886ns | evaluated request | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 263 |
| 2018-06-04 19:48:39.174538+00:00 | 1m29s386ms481µs554ns | acquired {raft,replica}mu | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 263 |
| 2018-06-04 19:48:39.174891+00:00 | 1m29s386ms834µs600ns | retry proposal 158d33b63000930a: reasonNewLeaderOrConfigChange | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 263 |
| 2018-06-04 19:48:39.174933+00:00 | 1m29s386ms876µs312ns | applied timestamp cache | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 263 |
| 2018-06-04 19:48:39.175024+00:00 | 1m29s386ms967µs162ns | evaluated request | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 263 |
| 2018-06-04 19:48:39.175047+00:00 | 1m29s386ms990µs463ns | acquired {raft,replica}mu | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 263 |
| 2018-06-04 19:48:39.175929+00:00 | 1m29s387ms872µs428ns | command proposed from replica (n1,s1):1 with lease #5 incompatible | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 263 |
| | | to repl=(n1,s1):1 seq=6 start=1528141568.368896594,0 epo=3 | | | | |
| | | pro=1528141715.778355836,0 | | | | |
| 2018-06-04 19:48:39.175949+00:00 | 1m29s387ms892µs17ns | applying command with forced error: cannot replace lease repl=(n1,s1):1 | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 263 |
| | | seq=6 start=1528141568.368896594,0 epo=3 pro=1528141715.778355836,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.368896594,0 epo=3 | | | | |
| | | pro=1528141715.778355836,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:39.176039+00:00 | 1m29s387ms982µs715ns | replica.Send got error: cannot replace lease repl=(n1,s1):1 seq=6 | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | [async] storage.pendingLeaseRequest: requesting lease | 263 |
| | | start=1528141568.368896594,0 epo=3 pro=1528141715.778355836,0 | | | | |
| | | with repl=(n1,s1):1 seq=6 start=1528141568.368896594,0 epo=3 | | | | |
| | | pro=1528141715.778355836,0: proposed under invalid lease | | | | |
| 2018-06-04 19:48:39.176095+00:00 | 1m29s388ms38µs163ns | read has no clock uncertainty | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | /cockroach.roachpb.Internal/Batch | 261 |
| 2018-06-04 19:48:39.176101+00:00 | 1m29s388ms43µs978ns | command queue | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | /cockroach.roachpb.Internal/Batch | 261 |
| 2018-06-04 19:48:39.176108+00:00 | 1m29s388ms51µs645ns | waiting for read lock | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | /cockroach.roachpb.Internal/Batch | 261 |
| 2018-06-04 19:48:39.176128+00:00 | 1m29s388ms70µs947ns | read completed | [n1,s1,r71/1:/Table/52/1/4{700-800}] | | /cockroach.roachpb.Internal/Batch | 261 |
| 2018-06-04 19:48:39.176281+00:00 | 1m29s388ms224µs233ns | === SPAN START: dist sender === | | | dist sender | 267 |
| 2018-06-04 19:48:39.176373+00:00 | 1m29s388ms315µs862ns | querying next range at /Table/52/1/4800 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 267 |
| 2018-06-04 19:48:39.17641+00:00 | 1m29s388ms353µs500ns | r72: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 267 |
| 2018-06-04 19:48:39.176429+00:00 | 1m29s388ms372µs271ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 267 |
| 2018-06-04 19:48:39.176446+00:00 | 1m29s388ms388µs831ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 268 |
| 2018-06-04 19:48:39.17645+00:00 | 1m29s388ms393µs660ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 268 |
| 2018-06-04 19:48:39.176456+00:00 | 1m29s388ms399µs794ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 268 |
| 2018-06-04 19:48:39.176463+00:00 | 1m29s388ms406µs763ns | read-only path | [n1,s1,r72/1:/Table/52/1/4{800-900}] | | /cockroach.roachpb.Internal/Batch | 268 |
| 2018-06-04 19:48:39.176485+00:00 | 1m29s388ms428µs151ns | replica.Send got error: [NotLeaseHolderError] r72: replica | [n1,s1,r72/1:/Table/52/1/4{800-900}] | | /cockroach.roachpb.Internal/Batch | 268 |
| | | (n1,s1):1 not lease holder; current lease is repl=(n3,s3):2 seq=4 | | | | |
| | | start=1528141715.924066503,0 epo=2 pro=1528141715.924070784,0 | | | | |
| 2018-06-04 19:48:39.176496+00:00 | 1m29s388ms439µs678ns | *roachpb.NotLeaseHolderError | [n1] | | /cockroach.roachpb.Internal/Batch | 268 |
| 2018-06-04 19:48:39.176515+00:00 | 1m29s388ms458µs370ns | application error: [NotLeaseHolderError] r72: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 267 |
| | | not lease holder; current lease is repl=(n3,s3):2 seq=4 | | | | |
| | | start=1528141715.924066503,0 epo=2 pro=1528141715.924070784,0 | | | | |
| 2018-06-04 19:48:39.176526+00:00 | 1m29s388ms469µs220ns | error: {(err: [NotLeaseHolderError] r72: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 267 |
| | | not lease holder; current lease is repl=(n3,s3):2 seq=4 | | | | |
| | | start=1528141715.924066503,0 epo=2 pro=1528141715.924070784,0) <nil>}; | | | | |
| | | trying next peer (n3,s3):2 | | | | |
| 2018-06-04 19:48:39.176542+00:00 | 1m29s388ms485µs569ns | sending request to alex-laptop.local:26261 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 267 |
| 2018-06-04 19:48:39.176566+00:00 | 1m29s388ms509µs252ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 269 |
| 2018-06-04 19:48:39.177012+00:00 | 1m29s388ms955µs497ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 446 |
| 2018-06-04 19:48:39.177086+00:00 | 1m29s389ms29µs655ns | 1 LeaseInfo | [n3] | | /cockroach.roachpb.Internal/Batch | 446 |
| 2018-06-04 19:48:39.177093+00:00 | 1m29s389ms36µs109ns | executing 1 requests | [n3,s3] | | /cockroach.roachpb.Internal/Batch | 446 |
| 2018-06-04 19:48:39.1771+00:00 | 1m29s389ms42µs867ns | read-only path | [n3,s3,r72/2:/Table/52/1/4{800-900}] | | /cockroach.roachpb.Internal/Batch | 446 |
| 2018-06-04 19:48:39.177107+00:00 | 1m29s389ms50µs450ns | command queue | [n3,s3,r72/2:/Table/52/1/4{800-900}] | | /cockroach.roachpb.Internal/Batch | 446 |
| 2018-06-04 19:48:39.177114+00:00 | 1m29s389ms56µs878ns | waiting for read lock | [n3,s3,r72/2:/Table/52/1/4{800-900}] | | /cockroach.roachpb.Internal/Batch | 446 |
| 2018-06-04 19:48:39.177123+00:00 | 1m29s389ms66µs571ns | read completed | [n3,s3,r72/2:/Table/52/1/4{800-900}] | | /cockroach.roachpb.Internal/Batch | 446 |
| 2018-06-04 19:48:39.177385+00:00 | 1m29s389ms328µs487ns | === SPAN START: dist sender === | | | dist sender | 270 |
| 2018-06-04 19:48:39.177397+00:00 | 1m29s389ms340µs110ns | querying next range at /Table/52/1/4900 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 270 |
| 2018-06-04 19:48:39.177414+00:00 | 1m29s389ms357µs373ns | r73: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 270 |
| 2018-06-04 19:48:39.177434+00:00 | 1m29s389ms377µs658ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 270 |
| 2018-06-04 19:48:39.177448+00:00 | 1m29s389ms390µs915ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 271 |
| 2018-06-04 19:48:39.177452+00:00 | 1m29s389ms395µs485ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 271 |
| 2018-06-04 19:48:39.177457+00:00 | 1m29s389ms400µs605ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 271 |
| 2018-06-04 19:48:39.177463+00:00 | 1m29s389ms406µs24ns | read-only path | [n1,s1,r73/1:/Table/52/1/{4900-5000}] | | /cockroach.roachpb.Internal/Batch | 271 |
| 2018-06-04 19:48:39.177483+00:00 | 1m29s389ms425µs958ns | replica.Send got error: [NotLeaseHolderError] r73: replica | [n1,s1,r73/1:/Table/52/1/{4900-5000}] | | /cockroach.roachpb.Internal/Batch | 271 |
| | | (n1,s1):1 not lease holder; current lease is repl=(n2,s2):3 seq=4 | | | | |
| | | start=1528141712.782373828,0 epo=2 pro=1528141712.782376306,0 | | | | |
| 2018-06-04 19:48:39.177493+00:00 | 1m29s389ms436µs692ns | *roachpb.NotLeaseHolderError | [n1] | | /cockroach.roachpb.Internal/Batch | 271 |
| 2018-06-04 19:48:39.177505+00:00 | 1m29s389ms448µs619ns | application error: [NotLeaseHolderError] r73: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 270 |
| | | not lease holder; current lease is repl=(n2,s2):3 seq=4 | | | | |
| | | start=1528141712.782373828,0 epo=2 pro=1528141712.782376306,0 | | | | |
| 2018-06-04 19:48:39.17753+00:00 | 1m29s389ms473µs15ns | error: {(err: [NotLeaseHolderError] r73: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 270 |
| | | not lease holder; current lease is repl=(n2,s2):3 seq=4 | | | | |
| | | start=1528141712.782373828,0 epo=2 pro=1528141712.782376306,0) <nil>}; | | | | |
| | | trying next peer (n2,s2):3 | | | | |
| 2018-06-04 19:48:39.177579+00:00 | 1m29s389ms522µs278ns | sending request to alex-laptop.local:26259 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 270 |
| 2018-06-04 19:48:39.177601+00:00 | 1m29s389ms544µs677ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 272 |
| 2018-06-04 19:48:39.177987+00:00 | 1m29s389ms930µs474ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 447 |
| 2018-06-04 19:48:39.178035+00:00 | 1m29s389ms978µs152ns | 1 LeaseInfo | [n2] | | /cockroach.roachpb.Internal/Batch | 447 |
| 2018-06-04 19:48:39.178047+00:00 | 1m29s389ms990µs440ns | executing 1 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 447 |
| 2018-06-04 19:48:39.178055+00:00 | 1m29s389ms998µs19ns | read-only path | [n2,s2,r73/3:/Table/52/1/{4900-5000}] | | /cockroach.roachpb.Internal/Batch | 447 |
| 2018-06-04 19:48:39.178064+00:00 | 1m29s390ms7µs163ns | command queue | [n2,s2,r73/3:/Table/52/1/{4900-5000}] | | /cockroach.roachpb.Internal/Batch | 447 |
| 2018-06-04 19:48:39.178069+00:00 | 1m29s390ms12µs435ns | waiting for read lock | [n2,s2,r73/3:/Table/52/1/{4900-5000}] | | /cockroach.roachpb.Internal/Batch | 447 |
| 2018-06-04 19:48:39.178082+00:00 | 1m29s390ms25µs360ns | read completed | [n2,s2,r73/3:/Table/52/1/{4900-5000}] | | /cockroach.roachpb.Internal/Batch | 447 |
| 2018-06-04 19:48:39.178498+00:00 | 1m29s390ms441µs286ns | === SPAN START: dist sender === | | | dist sender | 273 |
| 2018-06-04 19:48:39.178531+00:00 | 1m29s390ms474µs342ns | querying next range at /Table/52/1/5000 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 273 |
| 2018-06-04 19:48:39.178591+00:00 | 1m29s390ms534µs746ns | r74: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 273 |
| 2018-06-04 19:48:39.178644+00:00 | 1m29s390ms587µs296ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 273 |
| 2018-06-04 19:48:39.178698+00:00 | 1m29s390ms641µs430ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 274 |
| 2018-06-04 19:48:39.178705+00:00 | 1m29s390ms647µs880ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 274 |
| 2018-06-04 19:48:39.178711+00:00 | 1m29s390ms654µs274ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 274 |
| 2018-06-04 19:48:39.178716+00:00 | 1m29s390ms659µs425ns | read-only path | [n1,s1,r74/1:/Table/52/1/5{000-100}] | | /cockroach.roachpb.Internal/Batch | 274 |
| 2018-06-04 19:48:39.178721+00:00 | 1m29s390ms663µs877ns | request range lease (attempt #1) | [n1,s1,r74/1:/Table/52/1/5{000-100}] | | /cockroach.roachpb.Internal/Batch | 274 |
| 2018-06-04 19:48:44.179428+00:00 | 1m34s391ms371µs694ns | read has no clock uncertainty | [n1,s1,r74/1:/Table/52/1/5{000-100}] | | /cockroach.roachpb.Internal/Batch | 274 |
| 2018-06-04 19:48:44.179436+00:00 | 1m34s391ms379µs480ns | command queue | [n1,s1,r74/1:/Table/52/1/5{000-100}] | | /cockroach.roachpb.Internal/Batch | 274 |
| 2018-06-04 19:48:44.179445+00:00 | 1m34s391ms388µs595ns | waiting for read lock | [n1,s1,r74/1:/Table/52/1/5{000-100}] | | /cockroach.roachpb.Internal/Batch | 274 |
| 2018-06-04 19:48:44.179462+00:00 | 1m34s391ms405µs508ns | read completed | [n1,s1,r74/1:/Table/52/1/5{000-100}] | | /cockroach.roachpb.Internal/Batch | 274 |
| 2018-06-04 19:48:44.179565+00:00 | 1m34s391ms508µs525ns | === SPAN START: dist sender === | | | dist sender | 275 |
| 2018-06-04 19:48:44.179583+00:00 | 1m34s391ms525µs836ns | querying next range at /Table/52/1/5100 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 275 |
| 2018-06-04 19:48:44.179615+00:00 | 1m34s391ms558µs173ns | r75: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 275 |
| 2018-06-04 19:48:44.179637+00:00 | 1m34s391ms580µs730ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 275 |
| 2018-06-04 19:48:44.179652+00:00 | 1m34s391ms594µs902ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 276 |
| 2018-06-04 19:48:44.179656+00:00 | 1m34s391ms599µs42ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 276 |
| 2018-06-04 19:48:44.179661+00:00 | 1m34s391ms604µs497ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 276 |
| 2018-06-04 19:48:44.179667+00:00 | 1m34s391ms610µs685ns | read-only path | [n1,s1,r75/1:/Table/52/1/5{100-200}] | | /cockroach.roachpb.Internal/Batch | 276 |
| 2018-06-04 19:48:44.179684+00:00 | 1m34s391ms627µs518ns | replica.Send got error: [NotLeaseHolderError] r75: replica | [n1,s1,r75/1:/Table/52/1/5{100-200}] | | /cockroach.roachpb.Internal/Batch | 276 |
| | | (n1,s1):1 not lease holder; current lease is repl=(n3,s3):2 seq=6 | | | | |
| | | start=1528141716.382532425,0 epo=2 pro=1528141716.382537502,0 | | | | |
| 2018-06-04 19:48:44.179694+00:00 | 1m34s391ms637µs289ns | *roachpb.NotLeaseHolderError | [n1] | | /cockroach.roachpb.Internal/Batch | 276 |
| 2018-06-04 19:48:44.17971+00:00 | 1m34s391ms653µs196ns | application error: [NotLeaseHolderError] r75: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 275 |
| | | not lease holder; current lease is repl=(n3,s3):2 seq=6 | | | | |
| | | start=1528141716.382532425,0 epo=2 pro=1528141716.382537502,0 | | | | |
| 2018-06-04 19:48:44.17972+00:00 | 1m34s391ms662µs921ns | error: {(err: [NotLeaseHolderError] r75: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 275 |
| | | not lease holder; current lease is repl=(n3,s3):2 seq=6 | | | | |
| | | start=1528141716.382532425,0 epo=2 pro=1528141716.382537502,0) <nil>}; | | | | |
| | | trying next peer (n3,s3):2 | | | | |
| 2018-06-04 19:48:44.179735+00:00 | 1m34s391ms677µs840ns | sending request to alex-laptop.local:26261 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 275 |
| 2018-06-04 19:48:44.179757+00:00 | 1m34s391ms700µs334ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 277 |
| 2018-06-04 19:48:44.180061+00:00 | 1m34s392ms4µs148ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 448 |
| 2018-06-04 19:48:44.180096+00:00 | 1m34s392ms39µs534ns | 1 LeaseInfo | [n3] | | /cockroach.roachpb.Internal/Batch | 448 |
| 2018-06-04 19:48:44.180104+00:00 | 1m34s392ms47µs469ns | executing 1 requests | [n3,s3] | | /cockroach.roachpb.Internal/Batch | 448 |
| 2018-06-04 19:48:44.180112+00:00 | 1m34s392ms55µs222ns | read-only path | [n3,s3,r75/2:/Table/52/1/5{100-200}] | | /cockroach.roachpb.Internal/Batch | 448 |
| 2018-06-04 19:48:44.18012+00:00 | 1m34s392ms63µs404ns | command queue | [n3,s3,r75/2:/Table/52/1/5{100-200}] | | /cockroach.roachpb.Internal/Batch | 448 |
| 2018-06-04 19:48:44.180127+00:00 | 1m34s392ms70µs180ns | waiting for read lock | [n3,s3,r75/2:/Table/52/1/5{100-200}] | | /cockroach.roachpb.Internal/Batch | 448 |
| 2018-06-04 19:48:44.180141+00:00 | 1m34s392ms84µs333ns | read completed | [n3,s3,r75/2:/Table/52/1/5{100-200}] | | /cockroach.roachpb.Internal/Batch | 448 |
| 2018-06-04 19:48:44.180483+00:00 | 1m34s392ms426µs580ns | === SPAN START: dist sender === | | | dist sender | 278 |
| 2018-06-04 19:48:44.18053+00:00 | 1m34s392ms472µs961ns | querying next range at /Table/52/1/5200 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 278 |
| 2018-06-04 19:48:44.180561+00:00 | 1m34s392ms503µs880ns | r76: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 278 |
| 2018-06-04 19:48:44.180581+00:00 | 1m34s392ms524µs381ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 278 |
| 2018-06-04 19:48:44.180593+00:00 | 1m34s392ms536µs441ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 279 |
| 2018-06-04 19:48:44.180597+00:00 | 1m34s392ms540µs264ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 279 |
| 2018-06-04 19:48:44.180602+00:00 | 1m34s392ms545µs591ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 279 |
| 2018-06-04 19:48:44.180608+00:00 | 1m34s392ms550µs923ns | read-only path | [n1,s1,r76/1:/Table/52/1/5{200-300}] | | /cockroach.roachpb.Internal/Batch | 279 |
| 2018-06-04 19:48:44.180626+00:00 | 1m34s392ms569µs295ns | replica.Send got error: [NotLeaseHolderError] r76: replica | [n1,s1,r76/1:/Table/52/1/5{200-300}] | | /cockroach.roachpb.Internal/Batch | 279 |
| | | (n1,s1):1 not lease holder; current lease is repl=(n2,s2):3 seq=6 | | | | |
| | | start=1528141715.923999313,0 epo=2 pro=1528141715.924002790,0 | | | | |
| 2018-06-04 19:48:44.180636+00:00 | 1m34s392ms579µs14ns | *roachpb.NotLeaseHolderError | [n1] | | /cockroach.roachpb.Internal/Batch | 279 |
| 2018-06-04 19:48:44.180653+00:00 | 1m34s392ms596µs13ns | application error: [NotLeaseHolderError] r76: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 278 |
| | | not lease holder; current lease is repl=(n2,s2):3 seq=6 | | | | |
| | | start=1528141715.923999313,0 epo=2 pro=1528141715.924002790,0 | | | | |
| 2018-06-04 19:48:44.180663+00:00 | 1m34s392ms606µs593ns | error: {(err: [NotLeaseHolderError] r76: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 278 |
| | | not lease holder; current lease is repl=(n2,s2):3 seq=6 | | | | |
| | | start=1528141715.923999313,0 epo=2 pro=1528141715.924002790,0) <nil>}; | | | | |
| | | trying next peer (n2,s2):3 | | | | |
| 2018-06-04 19:48:44.180684+00:00 | 1m34s392ms627µs180ns | sending request to alex-laptop.local:26259 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 278 |
| 2018-06-04 19:48:44.180702+00:00 | 1m34s392ms645µs638ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 280 |
| 2018-06-04 19:48:44.180988+00:00 | 1m34s392ms931µs53ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 449 |
| 2018-06-04 19:48:44.181016+00:00 | 1m34s392ms959µs467ns | 1 LeaseInfo | [n2] | | /cockroach.roachpb.Internal/Batch | 449 |
| 2018-06-04 19:48:44.181023+00:00 | 1m34s392ms966µs360ns | executing 1 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 449 |
| 2018-06-04 19:48:44.18103+00:00 | 1m34s392ms973µs615ns | read-only path | [n2,s2,r76/3:/Table/52/1/5{200-300}] | | /cockroach.roachpb.Internal/Batch | 449 |
| 2018-06-04 19:48:44.181038+00:00 | 1m34s392ms981µs260ns | command queue | [n2,s2,r76/3:/Table/52/1/5{200-300}] | | /cockroach.roachpb.Internal/Batch | 449 |
| 2018-06-04 19:48:44.181043+00:00 | 1m34s392ms986µs294ns | waiting for read lock | [n2,s2,r76/3:/Table/52/1/5{200-300}] | | /cockroach.roachpb.Internal/Batch | 449 |
| 2018-06-04 19:48:44.181068+00:00 | 1m34s393ms11µs171ns | read completed | [n2,s2,r76/3:/Table/52/1/5{200-300}] | | /cockroach.roachpb.Internal/Batch | 449 |
| 2018-06-04 19:48:44.18147+00:00 | 1m34s393ms412µs909ns | === SPAN START: dist sender === | | | dist sender | 281 |
| 2018-06-04 19:48:44.181483+00:00 | 1m34s393ms426µs140ns | querying next range at /Table/52/1/5300 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 281 |
| 2018-06-04 19:48:44.181507+00:00 | 1m34s393ms450µs74ns | r77: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 281 |
| 2018-06-04 19:48:44.18153+00:00 | 1m34s393ms473µs278ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 281 |
| 2018-06-04 19:48:44.181548+00:00 | 1m34s393ms490µs979ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 282 |
| 2018-06-04 19:48:44.181553+00:00 | 1m34s393ms496µs784ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 282 |
| 2018-06-04 19:48:44.18156+00:00 | 1m34s393ms503µs432ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 282 |
| 2018-06-04 19:48:44.181567+00:00 | 1m34s393ms510µs2ns | read-only path | [n1,s1,r77/1:/Table/52/1/5{300-400}] | | /cockroach.roachpb.Internal/Batch | 282 |
| 2018-06-04 19:48:44.181573+00:00 | 1m34s393ms516µs362ns | request range lease (attempt #1) | [n1,s1,r77/1:/Table/52/1/5{300-400}] | | /cockroach.roachpb.Internal/Batch | 282 |
| 2018-06-04 19:48:47.179094+00:00 | 1m37s391ms37µs86ns | read has no clock uncertainty | [n1,s1,r77/1:/Table/52/1/5{300-400}] | | /cockroach.roachpb.Internal/Batch | 282 |
| 2018-06-04 19:48:47.179101+00:00 | 1m37s391ms44µs286ns | command queue | [n1,s1,r77/1:/Table/52/1/5{300-400}] | | /cockroach.roachpb.Internal/Batch | 282 |
| 2018-06-04 19:48:47.17911+00:00 | 1m37s391ms53µs414ns | waiting for read lock | [n1,s1,r77/1:/Table/52/1/5{300-400}] | | /cockroach.roachpb.Internal/Batch | 282 |
| 2018-06-04 19:48:47.179137+00:00 | 1m37s391ms80µs629ns | read completed | [n1,s1,r77/1:/Table/52/1/5{300-400}] | | /cockroach.roachpb.Internal/Batch | 282 |
| 2018-06-04 19:48:47.179231+00:00 | 1m37s391ms174µs717ns | === SPAN START: dist sender === | | | dist sender | 283 |
| 2018-06-04 19:48:47.179247+00:00 | 1m37s391ms190µs337ns | querying next range at /Table/52/1/5400 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 283 |
| 2018-06-04 19:48:47.179279+00:00 | 1m37s391ms222µs616ns | r78: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 283 |
| 2018-06-04 19:48:47.179297+00:00 | 1m37s391ms240µs32ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 283 |
| 2018-06-04 19:48:47.179309+00:00 | 1m37s391ms252µs741ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 284 |
| 2018-06-04 19:48:47.179313+00:00 | 1m37s391ms256µs544ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 284 |
| 2018-06-04 19:48:47.179318+00:00 | 1m37s391ms261µs722ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 284 |
| 2018-06-04 19:48:47.179324+00:00 | 1m37s391ms267µs787ns | read-only path | [n1,s1,r78/1:/Table/52/1/5{400-500}] | | /cockroach.roachpb.Internal/Batch | 284 |
| 2018-06-04 19:48:47.179342+00:00 | 1m37s391ms285µs656ns | replica.Send got error: [NotLeaseHolderError] r78: replica | [n1,s1,r78/1:/Table/52/1/5{400-500}] | | /cockroach.roachpb.Internal/Batch | 284 |
| | | (n1,s1):1 not lease holder; current lease is repl=(n3,s3):2 seq=6 | | | | |
| | | start=1528141719.775159447,0 epo=2 pro=1528141719.775164018,0 | | | | |
| 2018-06-04 19:48:47.179352+00:00 | 1m37s391ms294µs940ns | *roachpb.NotLeaseHolderError | [n1] | | /cockroach.roachpb.Internal/Batch | 284 |
| 2018-06-04 19:48:47.179368+00:00 | 1m37s391ms311µs675ns | application error: [NotLeaseHolderError] r78: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 283 |
| | | not lease holder; current lease is repl=(n3,s3):2 seq=6 | | | | |
| | | start=1528141719.775159447,0 epo=2 pro=1528141719.775164018,0 | | | | |
| 2018-06-04 19:48:47.179377+00:00 | 1m37s391ms320µs121ns | error: {(err: [NotLeaseHolderError] r78: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 283 |
| | | not lease holder; current lease is repl=(n3,s3):2 seq=6 | | | | |
| | | start=1528141719.775159447,0 epo=2 pro=1528141719.775164018,0) <nil>}; | | | | |
| | | trying next peer (n3,s3):2 | | | | |
| 2018-06-04 19:48:47.17939+00:00 | 1m37s391ms332µs995ns | sending request to alex-laptop.local:26261 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 283 |
| 2018-06-04 19:48:47.179413+00:00 | 1m37s391ms355µs913ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 285 |
| 2018-06-04 19:48:47.179655+00:00 | 1m37s391ms598µs35ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 450 |
| 2018-06-04 19:48:47.179683+00:00 | 1m37s391ms625µs838ns | 1 LeaseInfo | [n3] | | /cockroach.roachpb.Internal/Batch | 450 |
| 2018-06-04 19:48:47.179689+00:00 | 1m37s391ms632µs216ns | executing 1 requests | [n3,s3] | | /cockroach.roachpb.Internal/Batch | 450 |
| 2018-06-04 19:48:47.179696+00:00 | 1m37s391ms639µs133ns | read-only path | [n3,s3,r78/2:/Table/52/1/5{400-500}] | | /cockroach.roachpb.Internal/Batch | 450 |
| 2018-06-04 19:48:47.179703+00:00 | 1m37s391ms646µs125ns | command queue | [n3,s3,r78/2:/Table/52/1/5{400-500}] | | /cockroach.roachpb.Internal/Batch | 450 |
| 2018-06-04 19:48:47.179707+00:00 | 1m37s391ms650µs133ns | waiting for read lock | [n3,s3,r78/2:/Table/52/1/5{400-500}] | | /cockroach.roachpb.Internal/Batch | 450 |
| 2018-06-04 19:48:47.179719+00:00 | 1m37s391ms662µs567ns | read completed | [n3,s3,r78/2:/Table/52/1/5{400-500}] | | /cockroach.roachpb.Internal/Batch | 450 |
| 2018-06-04 19:48:47.179969+00:00 | 1m37s391ms911µs903ns | === SPAN START: dist sender === | | | dist sender | 286 |
| 2018-06-04 19:48:47.179979+00:00 | 1m37s391ms922µs463ns | querying next range at /Table/52/1/5500 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 286 |
| 2018-06-04 19:48:47.18+00:00 | 1m37s391ms943µs560ns | r79: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 286 |
| 2018-06-04 19:48:47.180017+00:00 | 1m37s391ms960µs742ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 286 |
| 2018-06-04 19:48:47.180028+00:00 | 1m37s391ms971µs68ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 287 |
| 2018-06-04 19:48:47.180031+00:00 | 1m37s391ms974µs212ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 287 |
| 2018-06-04 19:48:47.180035+00:00 | 1m37s391ms978µs400ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 287 |
| 2018-06-04 19:48:47.18004+00:00 | 1m37s391ms983µs639ns | read-only path | [n1,s1,r79/1:/Table/52/1/5{500-600}] | | /cockroach.roachpb.Internal/Batch | 287 |
| 2018-06-04 19:48:47.180057+00:00 | 1m37s392ms628ns | replica.Send got error: [NotLeaseHolderError] r79: replica | [n1,s1,r79/1:/Table/52/1/5{500-600}] | | /cockroach.roachpb.Internal/Batch | 287 |
| | | (n1,s1):1 not lease holder; current lease is repl=(n2,s2):3 seq=4 | | | | |
| | | start=1528141716.175438217,0 epo=2 pro=1528141716.175442204,0 | | | | |
| 2018-06-04 19:48:47.180066+00:00 | 1m37s392ms9µs332ns | *roachpb.NotLeaseHolderError | [n1] | | /cockroach.roachpb.Internal/Batch | 287 |
| 2018-06-04 19:48:47.180083+00:00 | 1m37s392ms26µs494ns | application error: [NotLeaseHolderError] r79: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 286 |
| | | not lease holder; current lease is repl=(n2,s2):3 seq=4 | | | | |
| | | start=1528141716.175438217,0 epo=2 pro=1528141716.175442204,0 | | | | |
| 2018-06-04 19:48:47.180093+00:00 | 1m37s392ms36µs721ns | error: {(err: [NotLeaseHolderError] r79: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 286 |
| | | not lease holder; current lease is repl=(n2,s2):3 seq=4 | | | | |
| | | start=1528141716.175438217,0 epo=2 pro=1528141716.175442204,0) <nil>}; | | | | |
| | | trying next peer (n2,s2):3 | | | | |
| 2018-06-04 19:48:47.180108+00:00 | 1m37s392ms51µs116ns | sending request to alex-laptop.local:26259 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 286 |
| 2018-06-04 19:48:47.180128+00:00 | 1m37s392ms71µs423ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 288 |
| 2018-06-04 19:48:47.180379+00:00 | 1m37s392ms322µs590ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 451 |
| 2018-06-04 19:48:47.180443+00:00 | 1m37s392ms386µs5ns | 1 LeaseInfo | [n2] | | /cockroach.roachpb.Internal/Batch | 451 |
| 2018-06-04 19:48:47.180451+00:00 | 1m37s392ms394µs758ns | executing 1 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 451 |
| 2018-06-04 19:48:47.180461+00:00 | 1m37s392ms403µs838ns | read-only path | [n2,s2,r79/3:/Table/52/1/5{500-600}] | | /cockroach.roachpb.Internal/Batch | 451 |
| 2018-06-04 19:48:47.180468+00:00 | 1m37s392ms411µs253ns | command queue | [n2,s2,r79/3:/Table/52/1/5{500-600}] | | /cockroach.roachpb.Internal/Batch | 451 |
| 2018-06-04 19:48:47.180473+00:00 | 1m37s392ms415µs833ns | waiting for read lock | [n2,s2,r79/3:/Table/52/1/5{500-600}] | | /cockroach.roachpb.Internal/Batch | 451 |
| 2018-06-04 19:48:47.180485+00:00 | 1m37s392ms428µs14ns | read completed | [n2,s2,r79/3:/Table/52/1/5{500-600}] | | /cockroach.roachpb.Internal/Batch | 451 |
| 2018-06-04 19:48:47.180848+00:00 | 1m37s392ms791µs100ns | === SPAN START: dist sender === | | | dist sender | 289 |
| 2018-06-04 19:48:47.180865+00:00 | 1m37s392ms807µs874ns | querying next range at /Table/52/1/5600 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 289 |
| 2018-06-04 19:48:47.180893+00:00 | 1m37s392ms836µs436ns | r80: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 289 |
| 2018-06-04 19:48:47.180912+00:00 | 1m37s392ms855µs413ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 289 |
| 2018-06-04 19:48:47.180923+00:00 | 1m37s392ms866µs291ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 290 |
| 2018-06-04 19:48:47.180926+00:00 | 1m37s392ms869µs799ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 290 |
| 2018-06-04 19:48:47.180931+00:00 | 1m37s392ms874µs75ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 290 |
| 2018-06-04 19:48:47.180936+00:00 | 1m37s392ms879µs176ns | read-only path | [n1,s1,r80/1:/Table/52/1/5{600-700}] | | /cockroach.roachpb.Internal/Batch | 290 |
| 2018-06-04 19:48:47.18094+00:00 | 1m37s392ms883µs479ns | request range lease (attempt #1) | [n1,s1,r80/1:/Table/52/1/5{600-700}] | | /cockroach.roachpb.Internal/Batch | 290 |
| 2018-06-04 19:48:50.775404+00:00 | 1m40s987ms347µs1ns | read has no clock uncertainty | [n1,s1,r80/1:/Table/52/1/5{600-700}] | | /cockroach.roachpb.Internal/Batch | 290 |
| 2018-06-04 19:48:50.775408+00:00 | 1m40s987ms351µs679ns | command queue | [n1,s1,r80/1:/Table/52/1/5{600-700}] | | /cockroach.roachpb.Internal/Batch | 290 |
| 2018-06-04 19:48:50.775415+00:00 | 1m40s987ms358µs44ns | waiting for read lock | [n1,s1,r80/1:/Table/52/1/5{600-700}] | | /cockroach.roachpb.Internal/Batch | 290 |
| 2018-06-04 19:48:50.775426+00:00 | 1m40s987ms369µs458ns | read completed | [n1,s1,r80/1:/Table/52/1/5{600-700}] | | /cockroach.roachpb.Internal/Batch | 290 |
| 2018-06-04 19:48:50.775508+00:00 | 1m40s987ms451µs45ns | === SPAN START: dist sender === | | | dist sender | 291 |
| 2018-06-04 19:48:50.775522+00:00 | 1m40s987ms465µs499ns | querying next range at /Table/52/1/5700 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 291 |
| 2018-06-04 19:48:50.775555+00:00 | 1m40s987ms498µs393ns | r81: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 291 |
| 2018-06-04 19:48:50.775572+00:00 | 1m40s987ms515µs381ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 291 |
| 2018-06-04 19:48:50.775585+00:00 | 1m40s987ms528µs11ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 292 |
| 2018-06-04 19:48:50.775588+00:00 | 1m40s987ms530µs952ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 292 |
| 2018-06-04 19:48:50.775595+00:00 | 1m40s987ms538µs723ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 292 |
| 2018-06-04 19:48:50.775611+00:00 | 1m40s987ms553µs897ns | read-only path | [n1,s1,r81/1:/Table/52/1/5{700-800}] | | /cockroach.roachpb.Internal/Batch | 292 |
| 2018-06-04 19:48:50.775625+00:00 | 1m40s987ms568µs658ns | replica.Send got error: [NotLeaseHolderError] r81: replica | [n1,s1,r81/1:/Table/52/1/5{700-800}] | | /cockroach.roachpb.Internal/Batch | 292 |
| | | (n1,s1):1 not lease holder; current lease is repl=(n3,s3):2 seq=4 | | | | |
| | | start=1528141722.983021047,0 epo=2 pro=1528141722.983025952,0 | | | | |
| 2018-06-04 19:48:50.775638+00:00 | 1m40s987ms581µs68ns | *roachpb.NotLeaseHolderError | [n1] | | /cockroach.roachpb.Internal/Batch | 292 |
| 2018-06-04 19:48:50.775651+00:00 | 1m40s987ms594µs201ns | application error: [NotLeaseHolderError] r81: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 291 |
| | | not lease holder; current lease is repl=(n3,s3):2 seq=4 | | | | |
| | | start=1528141722.983021047,0 epo=2 pro=1528141722.983025952,0 | | | | |
| 2018-06-04 19:48:50.775662+00:00 | 1m40s987ms605µs372ns | error: {(err: [NotLeaseHolderError] r81: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 291 |
| | | not lease holder; current lease is repl=(n3,s3):2 seq=4 | | | | |
| | | start=1528141722.983021047,0 epo=2 pro=1528141722.983025952,0) <nil>}; | | | | |
| | | trying next peer (n3,s3):2 | | | | |
| 2018-06-04 19:48:50.775673+00:00 | 1m40s987ms616µs589ns | sending request to alex-laptop.local:26261 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 291 |
| 2018-06-04 19:48:50.775699+00:00 | 1m40s987ms642µs630ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 293 |
| 2018-06-04 19:48:50.776025+00:00 | 1m40s987ms968µs602ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 452 |
| 2018-06-04 19:48:50.776063+00:00 | 1m40s988ms6µs377ns | 1 LeaseInfo | [n3] | | /cockroach.roachpb.Internal/Batch | 452 |
| 2018-06-04 19:48:50.77607+00:00 | 1m40s988ms13µs627ns | executing 1 requests | [n3,s3] | | /cockroach.roachpb.Internal/Batch | 452 |
| 2018-06-04 19:48:50.77608+00:00 | 1m40s988ms22µs926ns | read-only path | [n3,s3,r81/2:/Table/52/1/5{700-800}] | | /cockroach.roachpb.Internal/Batch | 452 |
| 2018-06-04 19:48:50.776089+00:00 | 1m40s988ms32µs31ns | command queue | [n3,s3,r81/2:/Table/52/1/5{700-800}] | | /cockroach.roachpb.Internal/Batch | 452 |
| 2018-06-04 19:48:50.776093+00:00 | 1m40s988ms36µs664ns | waiting for read lock | [n3,s3,r81/2:/Table/52/1/5{700-800}] | | /cockroach.roachpb.Internal/Batch | 452 |
| 2018-06-04 19:48:50.776106+00:00 | 1m40s988ms49µs33ns | read completed | [n3,s3,r81/2:/Table/52/1/5{700-800}] | | /cockroach.roachpb.Internal/Batch | 452 |
| 2018-06-04 19:48:50.776368+00:00 | 1m40s988ms311µs507ns | === SPAN START: dist sender === | | | dist sender | 294 |
| 2018-06-04 19:48:50.776378+00:00 | 1m40s988ms321µs526ns | querying next range at /Table/52/1/5800 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 294 |
| 2018-06-04 19:48:50.776396+00:00 | 1m40s988ms339µs117ns | r82: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 294 |
| 2018-06-04 19:48:50.776412+00:00 | 1m40s988ms355µs439ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 294 |
| 2018-06-04 19:48:50.776422+00:00 | 1m40s988ms365µs807ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 295 |
| 2018-06-04 19:48:50.776427+00:00 | 1m40s988ms370µs159ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 295 |
| 2018-06-04 19:48:50.776431+00:00 | 1m40s988ms374µs411ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 295 |
| 2018-06-04 19:48:50.776436+00:00 | 1m40s988ms379µs330ns | read-only path | [n1,s1,r82/1:/Table/52/1/5{800-900}] | | /cockroach.roachpb.Internal/Batch | 295 |
| 2018-06-04 19:48:50.776453+00:00 | 1m40s988ms396µs163ns | replica.Send got error: [NotLeaseHolderError] r82: replica | [n1,s1,r82/1:/Table/52/1/5{800-900}] | | /cockroach.roachpb.Internal/Batch | 295 |
| | | (n1,s1):1 not lease holder; current lease is repl=(n2,s2):3 seq=6 | | | | |
| | | start=1528141719.176882111,0 epo=2 pro=1528141719.176887570,0 | | | | |
| 2018-06-04 19:48:50.776466+00:00 | 1m40s988ms409µs131ns | *roachpb.NotLeaseHolderError | [n1] | | /cockroach.roachpb.Internal/Batch | 295 |
| 2018-06-04 19:48:50.776484+00:00 | 1m40s988ms427µs444ns | application error: [NotLeaseHolderError] r82: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 294 |
| | | not lease holder; current lease is repl=(n2,s2):3 seq=6 | | | | |
| | | start=1528141719.176882111,0 epo=2 pro=1528141719.176887570,0 | | | | |
| 2018-06-04 19:48:50.776493+00:00 | 1m40s988ms436µs2ns | error: {(err: [NotLeaseHolderError] r82: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 294 |
| | | not lease holder; current lease is repl=(n2,s2):3 seq=6 | | | | |
| | | start=1528141719.176882111,0 epo=2 pro=1528141719.176887570,0) <nil>}; | | | | |
| | | trying next peer (n2,s2):3 | | | | |
| 2018-06-04 19:48:50.776506+00:00 | 1m40s988ms449µs647ns | sending request to alex-laptop.local:26259 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 294 |
| 2018-06-04 19:48:50.77652+00:00 | 1m40s988ms463µs550ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 296 |
| 2018-06-04 19:48:50.776976+00:00 | 1m40s988ms919µs164ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 453 |
| 2018-06-04 19:48:50.777041+00:00 | 1m40s988ms984µs681ns | 1 LeaseInfo | [n2] | | /cockroach.roachpb.Internal/Batch | 453 |
| 2018-06-04 19:48:50.77705+00:00 | 1m40s988ms993µs391ns | executing 1 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 453 |
| 2018-06-04 19:48:50.777059+00:00 | 1m40s989ms2µs300ns | read-only path | [n2,s2,r82/3:/Table/52/1/5{800-900}] | | /cockroach.roachpb.Internal/Batch | 453 |
| 2018-06-04 19:48:50.777069+00:00 | 1m40s989ms12µs41ns | command queue | [n2,s2,r82/3:/Table/52/1/5{800-900}] | | /cockroach.roachpb.Internal/Batch | 453 |
| 2018-06-04 19:48:50.777075+00:00 | 1m40s989ms18µs306ns | waiting for read lock | [n2,s2,r82/3:/Table/52/1/5{800-900}] | | /cockroach.roachpb.Internal/Batch | 453 |
| 2018-06-04 19:48:50.777104+00:00 | 1m40s989ms47µs738ns | read completed | [n2,s2,r82/3:/Table/52/1/5{800-900}] | | /cockroach.roachpb.Internal/Batch | 453 |
| 2018-06-04 19:48:50.777423+00:00 | 1m40s989ms365µs988ns | === SPAN START: dist sender === | | | dist sender | 297 |
| 2018-06-04 19:48:50.777443+00:00 | 1m40s989ms386µs126ns | querying next range at /Table/52/1/5900 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 297 |
| 2018-06-04 19:48:50.777474+00:00 | 1m40s989ms417µs298ns | r83: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 297 |
| 2018-06-04 19:48:50.777501+00:00 | 1m40s989ms444µs536ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 297 |
| 2018-06-04 19:48:50.777521+00:00 | 1m40s989ms463µs977ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 298 |
| 2018-06-04 19:48:50.777526+00:00 | 1m40s989ms469µs315ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 298 |
| 2018-06-04 19:48:50.777533+00:00 | 1m40s989ms476µs553ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 298 |
| 2018-06-04 19:48:50.77754+00:00 | 1m40s989ms483µs584ns | read-only path | [n1,s1,r83/1:/Table/52/1/{5900-6000}] | | /cockroach.roachpb.Internal/Batch | 298 |
| 2018-06-04 19:48:50.777547+00:00 | 1m40s989ms490µs480ns | request range lease (attempt #1) | [n1,s1,r83/1:/Table/52/1/{5900-6000}] | | /cockroach.roachpb.Internal/Batch | 298 |
| 2018-06-04 19:48:54.375749+00:00 | 1m44s587ms691µs843ns | read has no clock uncertainty | [n1,s1,r83/1:/Table/52/1/{5900-6000}] | | /cockroach.roachpb.Internal/Batch | 298 |
| 2018-06-04 19:48:54.375757+00:00 | 1m44s587ms700µs582ns | command queue | [n1,s1,r83/1:/Table/52/1/{5900-6000}] | | /cockroach.roachpb.Internal/Batch | 298 |
| 2018-06-04 19:48:54.375767+00:00 | 1m44s587ms709µs887ns | waiting for read lock | [n1,s1,r83/1:/Table/52/1/{5900-6000}] | | /cockroach.roachpb.Internal/Batch | 298 |
| 2018-06-04 19:48:54.375784+00:00 | 1m44s587ms727µs250ns | read completed | [n1,s1,r83/1:/Table/52/1/{5900-6000}] | | /cockroach.roachpb.Internal/Batch | 298 |
| 2018-06-04 19:48:54.375908+00:00 | 1m44s587ms851µs191ns | === SPAN START: dist sender === | | | dist sender | 299 |
| 2018-06-04 19:48:54.375925+00:00 | 1m44s587ms868µs458ns | querying next range at /Table/52/1/6000 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 299 |
| 2018-06-04 19:48:54.37597+00:00 | 1m44s587ms913µs456ns | r84: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 299 |
| 2018-06-04 19:48:54.375995+00:00 | 1m44s587ms938µs451ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 299 |
| 2018-06-04 19:48:54.376011+00:00 | 1m44s587ms954µs104ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 300 |
| 2018-06-04 19:48:54.376016+00:00 | 1m44s587ms959µs27ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 300 |
| 2018-06-04 19:48:54.376021+00:00 | 1m44s587ms964µs585ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 300 |
| 2018-06-04 19:48:54.376028+00:00 | 1m44s587ms971µs621ns | read-only path | [n1,s1,r84/1:/Table/52/1/6{000-100}] | | /cockroach.roachpb.Internal/Batch | 300 |
| 2018-06-04 19:48:54.376047+00:00 | 1m44s587ms990µs125ns | replica.Send got error: [NotLeaseHolderError] r84: replica | [n1,s1,r84/1:/Table/52/1/6{000-100}] | | /cockroach.roachpb.Internal/Batch | 300 |
| | | (n1,s1):1 not lease holder; current lease is repl=(n3,s3):2 seq=6 | | | | |
| | | start=1528141727.975738894,0 epo=2 pro=1528141727.975745473,0 | | | | |
| 2018-06-04 19:48:54.376057+00:00 | 1m44s588ms436ns | *roachpb.NotLeaseHolderError | [n1] | | /cockroach.roachpb.Internal/Batch | 300 |
| 2018-06-04 19:48:54.376075+00:00 | 1m44s588ms17µs863ns | application error: [NotLeaseHolderError] r84: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 299 |
| | | not lease holder; current lease is repl=(n3,s3):2 seq=6 | | | | |
| | | start=1528141727.975738894,0 epo=2 pro=1528141727.975745473,0 | | | | |
| 2018-06-04 19:48:54.376085+00:00 | 1m44s588ms28µs263ns | error: {(err: [NotLeaseHolderError] r84: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 299 |
| | | not lease holder; current lease is repl=(n3,s3):2 seq=6 | | | | |
| | | start=1528141727.975738894,0 epo=2 pro=1528141727.975745473,0) <nil>}; | | | | |
| | | trying next peer (n3,s3):2 | | | | |
| 2018-06-04 19:48:54.376101+00:00 | 1m44s588ms43µs847ns | sending request to alex-laptop.local:26261 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 299 |
| 2018-06-04 19:48:54.376124+00:00 | 1m44s588ms67µs690ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 301 |
| 2018-06-04 19:48:54.376445+00:00 | 1m44s588ms388µs684ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 454 |
| 2018-06-04 19:48:54.376476+00:00 | 1m44s588ms419µs417ns | 1 LeaseInfo | [n3] | | /cockroach.roachpb.Internal/Batch | 454 |
| 2018-06-04 19:48:54.376483+00:00 | 1m44s588ms426µs692ns | executing 1 requests | [n3,s3] | | /cockroach.roachpb.Internal/Batch | 454 |
| 2018-06-04 19:48:54.376494+00:00 | 1m44s588ms437µs50ns | read-only path | [n3,s3,r84/2:/Table/52/1/6{000-100}] | | /cockroach.roachpb.Internal/Batch | 454 |
| 2018-06-04 19:48:54.376502+00:00 | 1m44s588ms444µs989ns | command queue | [n3,s3,r84/2:/Table/52/1/6{000-100}] | | /cockroach.roachpb.Internal/Batch | 454 |
| 2018-06-04 19:48:54.376506+00:00 | 1m44s588ms449µs729ns | waiting for read lock | [n3,s3,r84/2:/Table/52/1/6{000-100}] | | /cockroach.roachpb.Internal/Batch | 454 |
| 2018-06-04 19:48:54.37652+00:00 | 1m44s588ms463µs170ns | read completed | [n3,s3,r84/2:/Table/52/1/6{000-100}] | | /cockroach.roachpb.Internal/Batch | 454 |
| 2018-06-04 19:48:54.376915+00:00 | 1m44s588ms857µs970ns | === SPAN START: dist sender === | | | dist sender | 302 |
| 2018-06-04 19:48:54.37694+00:00 | 1m44s588ms883µs25ns | querying next range at /Table/52/1/6100 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 302 |
| 2018-06-04 19:48:54.376973+00:00 | 1m44s588ms915µs966ns | r85: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 302 |
| 2018-06-04 19:48:54.377+00:00 | 1m44s588ms943µs508ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 302 |
| 2018-06-04 19:48:54.377019+00:00 | 1m44s588ms962µs238ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 303 |
| 2018-06-04 19:48:54.377025+00:00 | 1m44s588ms968µs232ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 303 |
| 2018-06-04 19:48:54.377032+00:00 | 1m44s588ms975µs206ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 303 |
| 2018-06-04 19:48:54.37704+00:00 | 1m44s588ms983µs177ns | read-only path | [n1,s1,r85/1:/Table/52/1/6{100-200}] | | /cockroach.roachpb.Internal/Batch | 303 |
| 2018-06-04 19:48:54.377066+00:00 | 1m44s589ms8µs957ns | replica.Send got error: [NotLeaseHolderError] r85: replica | [n1,s1,r85/1:/Table/52/1/6{100-200}] | | /cockroach.roachpb.Internal/Batch | 303 |
| | | (n1,s1):1 not lease holder; current lease is repl=(n2,s2):3 seq=6 | | | | |
| | | start=1528141723.376944386,0 epo=2 pro=1528141723.376949091,0 | | | | |
| 2018-06-04 19:48:54.377081+00:00 | 1m44s589ms23µs971ns | *roachpb.NotLeaseHolderError | [n1] | | /cockroach.roachpb.Internal/Batch | 303 |
| 2018-06-04 19:48:54.377113+00:00 | 1m44s589ms56µs104ns | application error: [NotLeaseHolderError] r85: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 302 |
| | | not lease holder; current lease is repl=(n2,s2):3 seq=6 | | | | |
| | | start=1528141723.376944386,0 epo=2 pro=1528141723.376949091,0 | | | | |
| 2018-06-04 19:48:54.377124+00:00 | 1m44s589ms67µs456ns | error: {(err: [NotLeaseHolderError] r85: replica (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 302 |
| | | not lease holder; current lease is repl=(n2,s2):3 seq=6 | | | | |
| | | start=1528141723.376944386,0 epo=2 pro=1528141723.376949091,0) <nil>}; | | | | |
| | | trying next peer (n2,s2):3 | | | | |
| 2018-06-04 19:48:54.377141+00:00 | 1m44s589ms83µs949ns | sending request to alex-laptop.local:26259 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 302 |
| 2018-06-04 19:48:54.377162+00:00 | 1m44s589ms105µs301ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 304 |
| 2018-06-04 19:48:54.377462+00:00 | 1m44s589ms405µs250ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 455 |
| 2018-06-04 19:48:54.377499+00:00 | 1m44s589ms442µs317ns | 1 LeaseInfo | [n2] | | /cockroach.roachpb.Internal/Batch | 455 |
| 2018-06-04 19:48:54.377506+00:00 | 1m44s589ms449µs536ns | executing 1 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 455 |
| 2018-06-04 19:48:54.377513+00:00 | 1m44s589ms455µs977ns | read-only path | [n2,s2,r85/3:/Table/52/1/6{100-200}] | | /cockroach.roachpb.Internal/Batch | 455 |
| 2018-06-04 19:48:54.377529+00:00 | 1m44s589ms472µs276ns | command queue | [n2,s2,r85/3:/Table/52/1/6{100-200}] | | /cockroach.roachpb.Internal/Batch | 455 |
| 2018-06-04 19:48:54.377535+00:00 | 1m44s589ms478µs281ns | waiting for read lock | [n2,s2,r85/3:/Table/52/1/6{100-200}] | | /cockroach.roachpb.Internal/Batch | 455 |
| 2018-06-04 19:48:54.377545+00:00 | 1m44s589ms488µs155ns | read completed | [n2,s2,r85/3:/Table/52/1/6{100-200}] | | /cockroach.roachpb.Internal/Batch | 455 |
| 2018-06-04 19:48:54.37787+00:00 | 1m44s589ms812µs978ns | === SPAN START: dist sender === | | | dist sender | 305 |
| 2018-06-04 19:48:54.37789+00:00 | 1m44s589ms832µs972ns | querying next range at /Table/52/1/6200 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 305 |
| 2018-06-04 19:48:54.377921+00:00 | 1m44s589ms863µs860ns | r86: sending batch 1 LeaseInfo to (n1,s1):1 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 305 |
| 2018-06-04 19:48:54.377946+00:00 | 1m44s589ms889µs595ns | sending request to local server | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 305 |
| 2018-06-04 19:48:54.377959+00:00 | 1m44s589ms902µs27ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 306 |
| 2018-06-04 19:48:54.377963+00:00 | 1m44s589ms906µs765ns | 1 LeaseInfo | [n1] | | /cockroach.roachpb.Internal/Batch | 306 |
| 2018-06-04 19:48:54.377968+00:00 | 1m44s589ms911µs371ns | executing 1 requests | [n1,s1] | | /cockroach.roachpb.Internal/Batch | 306 |
| 2018-06-04 19:48:54.377973+00:00 | 1m44s589ms916µs70ns | read-only path | [n1,s1,r86/1:/Table/52/1/6{200-300}] | | /cockroach.roachpb.Internal/Batch | 306 |
| 2018-06-04 19:48:54.377977+00:00 | 1m44s589ms919µs931ns | request range lease (attempt #1) | [n1,s1,r86/1:/Table/52/1/6{200-300}] | | /cockroach.roachpb.Internal/Batch | 306 |
| 2018-06-04 19:48:57.581048+00:00 | 1m47s792ms991µs169ns | read has no clock uncertainty | [n1,s1,r86/1:/Table/52/1/6{200-300}] | | /cockroach.roachpb.Internal/Batch | 306 |
| 2018-06-04 19:48:57.581054+00:00 | 1m47s792ms996µs881ns | command queue | [n1,s1,r86/1:/Table/52/1/6{200-300}] | | /cockroach.roachpb.Internal/Batch | 306 |
| 2018-06-04 19:48:57.581061+00:00 | 1m47s793ms4µs155ns | waiting for read lock | [n1,s1,r86/1:/Table/52/1/6{200-300}] | | /cockroach.roachpb.Internal/Batch | 306 |
| 2018-06-04 19:48:57.581074+00:00 | 1m47s793ms17µs65ns | read completed | [n1,s1,r86/1:/Table/52/1/6{200-300}] | | /cockroach.roachpb.Internal/Batch | 306 |
| 2018-06-04 19:48:57.581156+00:00 | 1m47s793ms99µs559ns | === SPAN START: dist sender === | | | dist sender | 307 |
| 2018-06-04 19:48:57.581172+00:00 | 1m47s793ms115µs415ns | querying next range at /Table/52/1/6300 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 307 |
| 2018-06-04 19:48:57.581206+00:00 | 1m47s793ms149µs177ns | r87: sending batch 1 LeaseInfo to (n3,s3):2 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 307 |
| 2018-06-04 19:48:57.581222+00:00 | 1m47s793ms165µs591ns | sending request to alex-laptop.local:26261 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 307 |
| 2018-06-04 19:48:57.581245+00:00 | 1m47s793ms188µs812ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 308 |
| 2018-06-04 19:48:57.581587+00:00 | 1m47s793ms530µs515ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 456 |
| 2018-06-04 19:48:57.581621+00:00 | 1m47s793ms564µs278ns | 1 LeaseInfo | [n3] | | /cockroach.roachpb.Internal/Batch | 456 |
| 2018-06-04 19:48:57.58163+00:00 | 1m47s793ms573µs605ns | executing 1 requests | [n3,s3] | | /cockroach.roachpb.Internal/Batch | 456 |
| 2018-06-04 19:48:57.581641+00:00 | 1m47s793ms584µs367ns | read-only path | [n3,s3,r87/2:/Table/52/1/6{300-400}] | | /cockroach.roachpb.Internal/Batch | 456 |
| 2018-06-04 19:48:57.581651+00:00 | 1m47s793ms594µs779ns | command queue | [n3,s3,r87/2:/Table/52/1/6{300-400}] | | /cockroach.roachpb.Internal/Batch | 456 |
| 2018-06-04 19:48:57.581658+00:00 | 1m47s793ms601µs465ns | waiting for read lock | [n3,s3,r87/2:/Table/52/1/6{300-400}] | | /cockroach.roachpb.Internal/Batch | 456 |
| 2018-06-04 19:48:57.58167+00:00 | 1m47s793ms613µs148ns | read completed | [n3,s3,r87/2:/Table/52/1/6{300-400}] | | /cockroach.roachpb.Internal/Batch | 456 |
| 2018-06-04 19:48:57.581954+00:00 | 1m47s793ms897µs591ns | === SPAN START: dist sender === | | | dist sender | 309 |
| 2018-06-04 19:48:57.581967+00:00 | 1m47s793ms910µs471ns | querying next range at /Table/52/1/6400 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 309 |
| 2018-06-04 19:48:57.581991+00:00 | 1m47s793ms934µs19ns | r88: sending batch 1 LeaseInfo to (n3,s3):2 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 309 |
| 2018-06-04 19:48:57.58201+00:00 | 1m47s793ms953µs60ns | sending request to alex-laptop.local:26261 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 309 |
| 2018-06-04 19:48:57.582025+00:00 | 1m47s793ms968µs385ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 310 |
| 2018-06-04 19:48:57.582251+00:00 | 1m47s794ms194µs336ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 457 |
| 2018-06-04 19:48:57.582287+00:00 | 1m47s794ms230µs211ns | 1 LeaseInfo | [n3] | | /cockroach.roachpb.Internal/Batch | 457 |
| 2018-06-04 19:48:57.582295+00:00 | 1m47s794ms237µs954ns | executing 1 requests | [n3,s3] | | /cockroach.roachpb.Internal/Batch | 457 |
| 2018-06-04 19:48:57.582302+00:00 | 1m47s794ms245µs764ns | read-only path | [n3,s3,r88/2:/Table/52/1/6{400-500}] | | /cockroach.roachpb.Internal/Batch | 457 |
| 2018-06-04 19:48:57.582372+00:00 | 1m47s794ms315µs251ns | replica.Send got error: [NotLeaseHolderError] r88: replica | [n3,s3,r88/2:/Table/52/1/6{400-500}] | | /cockroach.roachpb.Internal/Batch | 457 |
| | | (n3,s3):2 not lease holder; current lease is repl=(n2,s2):3 seq=6 | | | | |
| | | start=1528141726.779738033,0 epo=2 pro=1528141726.779746542,0 | | | | |
| 2018-06-04 19:48:57.582385+00:00 | 1m47s794ms328µs488ns | *roachpb.NotLeaseHolderError | [n3] | | /cockroach.roachpb.Internal/Batch | 457 |
| 2018-06-04 19:48:57.582678+00:00 | 1m47s794ms621µs643ns | application error: [NotLeaseHolderError] r88: replica (n3,s3):2 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 309 |
| | | not lease holder; current lease is repl=(n2,s2):3 seq=6 | | | | |
| | | start=1528141726.779738033,0 epo=2 pro=1528141726.779746542,0 | | | | |
| 2018-06-04 19:48:57.582694+00:00 | 1m47s794ms636µs876ns | error: {(err: [NotLeaseHolderError] r88: replica (n3,s3):2 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 309 |
| | | not lease holder; current lease is repl=(n2,s2):3 seq=6 | | | | |
| | | start=1528141726.779738033,0 epo=2 pro=1528141726.779746542,0) <nil>}; | | | | |
| | | trying next peer (n2,s2):3 | | | | |
| 2018-06-04 19:48:57.58271+00:00 | 1m47s794ms653µs508ns | sending request to alex-laptop.local:26259 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 309 |
| 2018-06-04 19:48:57.582732+00:00 | 1m47s794ms674µs974ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 311 |
| 2018-06-04 19:48:57.583076+00:00 | 1m47s795ms18µs976ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 458 |
| 2018-06-04 19:48:57.583119+00:00 | 1m47s795ms61µs950ns | 1 LeaseInfo | [n2] | | /cockroach.roachpb.Internal/Batch | 458 |
| 2018-06-04 19:48:57.583127+00:00 | 1m47s795ms70µs498ns | executing 1 requests | [n2,s2] | | /cockroach.roachpb.Internal/Batch | 458 |
| 2018-06-04 19:48:57.583136+00:00 | 1m47s795ms79µs711ns | read-only path | [n2,s2,r88/3:/Table/52/1/6{400-500}] | | /cockroach.roachpb.Internal/Batch | 458 |
| 2018-06-04 19:48:57.583147+00:00 | 1m47s795ms89µs933ns | command queue | [n2,s2,r88/3:/Table/52/1/6{400-500}] | | /cockroach.roachpb.Internal/Batch | 458 |
| 2018-06-04 19:48:57.583153+00:00 | 1m47s795ms96µs700ns | waiting for read lock | [n2,s2,r88/3:/Table/52/1/6{400-500}] | | /cockroach.roachpb.Internal/Batch | 458 |
| 2018-06-04 19:48:57.583168+00:00 | 1m47s795ms111µs481ns | read completed | [n2,s2,r88/3:/Table/52/1/6{400-500}] | | /cockroach.roachpb.Internal/Batch | 458 |
| 2018-06-04 19:48:57.583645+00:00 | 1m47s795ms588µs597ns | === SPAN START: dist sender === | | | dist sender | 312 |
| 2018-06-04 19:48:57.583658+00:00 | 1m47s795ms601µs459ns | querying next range at /Table/52/1/6500 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 312 |
| 2018-06-04 19:48:57.58368+00:00 | 1m47s795ms623µs662ns | r89: sending batch 1 LeaseInfo to (n3,s3):2 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 312 |
| 2018-06-04 19:48:57.583698+00:00 | 1m47s795ms641µs108ns | sending request to alex-laptop.local:26261 | [client=[::1]:57501,user=root,txn=9352587e,n1] | | dist sender | 312 |
| 2018-06-04 19:48:57.583713+00:00 | 1m47s795ms655µs918ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 313 |
| 2018-06-04 19:48:57.583886+00:00 | 1m47s795ms829µs159ns | === SPAN START: /cockroach.roachpb.Internal/Batch === | | | /cockroach.roachpb.Internal/Batch | 459 |
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment