Created
January 26, 2015 14:20
Revisions
-
lowang created this gist
Jan 26, 2015 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,629 @@ m30999| 2015-01-26T14:59:41.936+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31102| 2015-01-26T14:59:44.699+0100 [conn210] end connection 192.168.1.220:52935 (4 connections now open) m31102| 2015-01-26T14:59:44.701+0100 [initandlisten] connection accepted from 192.168.1.220:52951 #213 (5 connections now open) m31100| 2015-01-26T14:59:46.655+0100 [conn234] end connection 192.168.1.220:52936 (10 connections now open) m31100| 2015-01-26T14:59:46.656+0100 [initandlisten] connection accepted from 192.168.1.220:52952 #237 (11 connections now open) m30999| 2015-01-26T14:59:47.945+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c64853ce5c61cbab9c833b m30999| 2015-01-26T14:59:47.945+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m30999| 2015-01-26T14:59:50.723+0100 [LockPinger] cluster Lowang-MacBook-Pro.local:29000 pinged successfully at Mon Jan 26 14:59:50 2015 by distributed lock pinger 'Lowang-MacBook-Pro.local:29000/Lowang-MacBook-Pro.local:30999:1422278030:16807', sleeping for 30000ms m31101| 2015-01-26T14:59:52.105+0100 [conn213] end connection 192.168.59.3:52937 (6 connections now open) m31101| 2015-01-26T14:59:52.108+0100 [initandlisten] connection accepted from 192.168.59.3:52953 #216 (7 connections now open) m30999| 2015-01-26T14:59:53.960+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c64859ce5c61cbab9c833c m30999| 2015-01-26T14:59:53.960+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31100| 2015-01-26T14:59:54.227+0100 [conn235] end connection 192.168.59.3:52939 (10 connections now open) m31100| 2015-01-26T14:59:54.227+0100 [initandlisten] connection accepted from 192.168.59.3:52955 #238 (11 connections now open) m31100| 2015-01-26T14:59:54.364+0100 [conn236] end connection 192.168.59.3:52940 (10 connections now open) m31100| 2015-01-26T14:59:54.365+0100 [initandlisten] connection accepted from 192.168.59.3:52956 #239 (11 connections now open) m30999| 2015-01-26T14:59:59.975+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c6485fce5c61cbab9c833d m30999| 2015-01-26T14:59:59.975+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31101| 2015-01-26T15:00:00.695+0100 [conn214] end connection 192.168.1.220:52944 (6 connections now open) m31101| 2015-01-26T15:00:00.703+0100 [initandlisten] connection accepted from 192.168.1.220:52961 #217 (7 connections now open) m31101| 2015-01-26T15:00:04.043+0100 [conn14] command admin.$cmd command: writebacklisten { writebacklisten: ObjectId('54c63d8ece5c61cbab9c816b') } ntoreturn:1 keyUpdates:0 numYields:0 reslen:44 300008ms m30999| 2015-01-26T15:00:05.991+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c64865ce5c61cbab9c833e m30999| 2015-01-26T15:00:05.991+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31102| 2015-01-26T15:00:06.164+0100 [conn211] end connection 192.168.59.3:52945 (4 connections now open) m31102| 2015-01-26T15:00:06.164+0100 [initandlisten] connection accepted from 192.168.59.3:52962 #214 (5 connections now open) m31100| 2015-01-26T15:00:07.699+0100 [conn174] command admin.$cmd command: writebacklisten { writebacklisten: ObjectId('54c63d8ece5c61cbab9c816b') } ntoreturn:1 keyUpdates:0 numYields:0 reslen:44 300006ms m31102| 2015-01-26T15:00:08.259+0100 [conn212] end connection 192.168.59.3:52948 (4 connections now open) m31102| 2015-01-26T15:00:08.260+0100 [initandlisten] connection accepted from 192.168.59.3:52964 #215 (5 connections now open) m31101| 2015-01-26T15:00:08.422+0100 [conn215] end connection 192.168.59.3:52949 (6 connections now open) m31101| 2015-01-26T15:00:08.423+0100 [initandlisten] connection accepted from 192.168.59.3:52965 #218 (7 connections now open) m30999| 2015-01-26T15:00:12.006+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c6486cce5c61cbab9c833f m30999| 2015-01-26T15:00:12.006+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31102| 2015-01-26T15:00:14.785+0100 [conn213] end connection 192.168.1.220:52951 (4 connections now open) m31102| 2015-01-26T15:00:14.794+0100 [initandlisten] connection accepted from 192.168.1.220:52966 #216 (5 connections now open) m31100| 2015-01-26T15:00:16.745+0100 [conn237] end connection 192.168.1.220:52952 (10 connections now open) m31100| 2015-01-26T15:00:16.745+0100 [initandlisten] connection accepted from 192.168.1.220:52968 #240 (11 connections now open) m30999| 2015-01-26T15:00:18.024+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c64872ce5c61cbab9c8340 m30999| 2015-01-26T15:00:18.025+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m30999| 2015-01-26T15:00:20.732+0100 [LockPinger] cluster Lowang-MacBook-Pro.local:29000 pinged successfully at Mon Jan 26 15:00:20 2015 by distributed lock pinger 'Lowang-MacBook-Pro.local:29000/Lowang-MacBook-Pro.local:30999:1422278030:16807', sleeping for 30000ms m31101| 2015-01-26T15:00:22.228+0100 [conn216] end connection 192.168.59.3:52953 (6 connections now open) m31101| 2015-01-26T15:00:22.229+0100 [initandlisten] connection accepted from 192.168.59.3:52969 #219 (7 connections now open) m30999| 2015-01-26T15:00:24.046+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c64878ce5c61cbab9c8341 m30999| 2015-01-26T15:00:24.047+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31100| 2015-01-26T15:00:24.349+0100 [conn238] end connection 192.168.59.3:52955 (10 connections now open) m31100| 2015-01-26T15:00:24.350+0100 [initandlisten] connection accepted from 192.168.59.3:52971 #241 (11 connections now open) m31100| 2015-01-26T15:00:24.488+0100 [conn239] end connection 192.168.59.3:52956 (10 connections now open) m31100| 2015-01-26T15:00:24.489+0100 [initandlisten] connection accepted from 192.168.59.3:52972 #242 (11 connections now open) m30999| 2015-01-26T15:00:30.055+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c6487ece5c61cbab9c8342 m30999| 2015-01-26T15:00:30.055+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31101| 2015-01-26T15:00:30.769+0100 [conn217] end connection 192.168.1.220:52961 (6 connections now open) m31101| 2015-01-26T15:00:30.775+0100 [initandlisten] connection accepted from 192.168.1.220:52976 #220 (7 connections now open) m30999| 2015-01-26T15:00:36.065+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c64884ce5c61cbab9c8343 m30999| 2015-01-26T15:00:36.066+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31102| 2015-01-26T15:00:36.280+0100 [conn214] end connection 192.168.59.3:52962 (4 connections now open) m31102| 2015-01-26T15:00:36.281+0100 [initandlisten] connection accepted from 192.168.59.3:52977 #217 (5 connections now open) m31102| 2015-01-26T15:00:38.385+0100 [conn215] end connection 192.168.59.3:52964 (4 connections now open) m31102| 2015-01-26T15:00:38.386+0100 [initandlisten] connection accepted from 192.168.59.3:52980 #218 (5 connections now open) m31101| 2015-01-26T15:00:38.546+0100 [conn218] end connection 192.168.59.3:52965 (6 connections now open) m31101| 2015-01-26T15:00:38.547+0100 [initandlisten] connection accepted from 192.168.59.3:52981 #221 (7 connections now open) m30999| 2015-01-26T15:00:42.080+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c6488ace5c61cbab9c8344 m30999| 2015-01-26T15:00:42.081+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31102| 2015-01-26T15:00:44.877+0100 [conn216] end connection 192.168.1.220:52966 (4 connections now open) m31102| 2015-01-26T15:00:44.880+0100 [initandlisten] connection accepted from 192.168.1.220:52983 #219 (5 connections now open) m31100| 2015-01-26T15:00:46.800+0100 [conn240] end connection 192.168.1.220:52968 (10 connections now open) m31100| 2015-01-26T15:00:46.804+0100 [initandlisten] connection accepted from 192.168.1.220:52985 #243 (11 connections now open) m30999| 2015-01-26T15:00:48.095+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c64890ce5c61cbab9c8345 m30999| 2015-01-26T15:00:48.096+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m30999| 2015-01-26T15:00:50.741+0100 [LockPinger] cluster Lowang-MacBook-Pro.local:29000 pinged successfully at Mon Jan 26 15:00:50 2015 by distributed lock pinger 'Lowang-MacBook-Pro.local:29000/Lowang-MacBook-Pro.local:30999:1422278030:16807', sleeping for 30000ms m31101| 2015-01-26T15:00:52.346+0100 [conn219] end connection 192.168.59.3:52969 (6 connections now open) m31101| 2015-01-26T15:00:52.348+0100 [initandlisten] connection accepted from 192.168.59.3:52986 #222 (7 connections now open) m29000| 2015-01-26T15:00:52.643+0100 [clientcursormon] mem (MB) res:50 virt:2611 m29000| 2015-01-26T15:00:52.643+0100 [clientcursormon] mapped (incl journal view):128 m29000| 2015-01-26T15:00:52.644+0100 [clientcursormon] connections:9 m30999| 2015-01-26T15:00:54.107+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c64896ce5c61cbab9c8346 m30999| 2015-01-26T15:00:54.108+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31100| 2015-01-26T15:00:54.460+0100 [conn241] end connection 192.168.59.3:52971 (10 connections now open) m31100| 2015-01-26T15:00:54.461+0100 [initandlisten] connection accepted from 192.168.59.3:52988 #244 (11 connections now open) m31100| 2015-01-26T15:00:54.618+0100 [conn242] end connection 192.168.59.3:52972 (10 connections now open) m31100| 2015-01-26T15:00:54.619+0100 [initandlisten] connection accepted from 192.168.59.3:52989 #245 (11 connections now open) m30999| 2015-01-26T15:01:00.118+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c6489cce5c61cbab9c8347 m30999| 2015-01-26T15:01:00.119+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31101| 2015-01-26T15:01:00.841+0100 [conn220] end connection 192.168.1.220:52976 (6 connections now open) m31101| 2015-01-26T15:01:00.845+0100 [initandlisten] connection accepted from 192.168.1.220:52995 #223 (7 connections now open) m30999| 2015-01-26T15:01:06.128+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c648a2ce5c61cbab9c8348 m30999| 2015-01-26T15:01:06.129+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31102| 2015-01-26T15:01:06.407+0100 [conn217] end connection 192.168.59.3:52977 (4 connections now open) m31102| 2015-01-26T15:01:06.409+0100 [initandlisten] connection accepted from 192.168.59.3:52996 #220 (5 connections now open) m31102| 2015-01-26T15:01:08.490+0100 [conn218] end connection 192.168.59.3:52980 (4 connections now open) m31102| 2015-01-26T15:01:08.491+0100 [initandlisten] connection accepted from 192.168.59.3:52998 #221 (5 connections now open) m31101| 2015-01-26T15:01:08.675+0100 [conn221] end connection 192.168.59.3:52981 (6 connections now open) m31101| 2015-01-26T15:01:08.676+0100 [initandlisten] connection accepted from 192.168.59.3:52999 #224 (7 connections now open) m30999| 2015-01-26T15:01:12.141+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c648a8ce5c61cbab9c8349 m30999| 2015-01-26T15:01:12.142+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31102| 2015-01-26T15:01:14.961+0100 [conn219] end connection 192.168.1.220:52983 (4 connections now open) m31102| 2015-01-26T15:01:14.965+0100 [initandlisten] connection accepted from 192.168.1.220:53000 #222 (5 connections now open) m31100| 2015-01-26T15:01:16.874+0100 [conn243] end connection 192.168.1.220:52985 (10 connections now open) m31100| 2015-01-26T15:01:16.874+0100 [initandlisten] connection accepted from 192.168.1.220:53001 #246 (11 connections now open) m30999| 2015-01-26T15:01:18.154+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c648aece5c61cbab9c834a m30999| 2015-01-26T15:01:18.154+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m30999| 2015-01-26T15:01:20.750+0100 [LockPinger] cluster Lowang-MacBook-Pro.local:29000 pinged successfully at Mon Jan 26 15:01:20 2015 by distributed lock pinger 'Lowang-MacBook-Pro.local:29000/Lowang-MacBook-Pro.local:30999:1422278030:16807', sleeping for 30000ms m31101| 2015-01-26T15:01:22.467+0100 [conn222] end connection 192.168.59.3:52986 (6 connections now open) m31101| 2015-01-26T15:01:22.469+0100 [initandlisten] connection accepted from 192.168.59.3:53003 #225 (7 connections now open) m30999| 2015-01-26T15:01:24.167+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c648b4ce5c61cbab9c834b m30999| 2015-01-26T15:01:24.168+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31100| 2015-01-26T15:01:24.580+0100 [conn244] end connection 192.168.59.3:52988 (10 connections now open) m31100| 2015-01-26T15:01:24.581+0100 [initandlisten] connection accepted from 192.168.59.3:53005 #247 (11 connections now open) m31100| 2015-01-26T15:01:24.748+0100 [conn245] end connection 192.168.59.3:52989 (10 connections now open) m31100| 2015-01-26T15:01:24.749+0100 [initandlisten] connection accepted from 192.168.59.3:53006 #248 (11 connections now open) m30999| 2015-01-26T15:01:30.177+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c648bace5c61cbab9c834c m30999| 2015-01-26T15:01:30.178+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31101| 2015-01-26T15:01:30.897+0100 [conn223] end connection 192.168.1.220:52995 (6 connections now open) m31101| 2015-01-26T15:01:30.899+0100 [initandlisten] connection accepted from 192.168.1.220:53010 #226 (7 connections now open) m30999| 2015-01-26T15:01:36.190+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c648c0ce5c61cbab9c834d m30999| 2015-01-26T15:01:36.190+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31102| 2015-01-26T15:01:36.525+0100 [conn220] end connection 192.168.59.3:52996 (4 connections now open) m31102| 2015-01-26T15:01:36.526+0100 [initandlisten] connection accepted from 192.168.59.3:53011 #223 (5 connections now open) m31102| 2015-01-26T15:01:38.600+0100 [conn221] end connection 192.168.59.3:52998 (4 connections now open) m31102| 2015-01-26T15:01:38.601+0100 [initandlisten] connection accepted from 192.168.59.3:53013 #224 (5 connections now open) m31101| 2015-01-26T15:01:38.800+0100 [conn224] end connection 192.168.59.3:52999 (6 connections now open) m31101| 2015-01-26T15:01:38.800+0100 [initandlisten] connection accepted from 192.168.59.3:53014 #227 (7 connections now open) m30999| 2015-01-26T15:01:42.206+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c648c6ce5c61cbab9c834e m30999| 2015-01-26T15:01:42.207+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31102| 2015-01-26T15:01:45.051+0100 [conn222] end connection 192.168.1.220:53000 (4 connections now open) m31102| 2015-01-26T15:01:45.054+0100 [initandlisten] connection accepted from 192.168.1.220:53016 #225 (5 connections now open) m31100| 2015-01-26T15:01:46.932+0100 [conn246] end connection 192.168.1.220:53001 (10 connections now open) m31100| 2015-01-26T15:01:46.933+0100 [initandlisten] connection accepted from 192.168.1.220:53017 #249 (11 connections now open) m30999| 2015-01-26T15:01:48.222+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c648ccce5c61cbab9c834f m30999| 2015-01-26T15:01:48.222+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m30999| 2015-01-26T15:01:50.761+0100 [LockPinger] cluster Lowang-MacBook-Pro.local:29000 pinged successfully at Mon Jan 26 15:01:50 2015 by distributed lock pinger 'Lowang-MacBook-Pro.local:29000/Lowang-MacBook-Pro.local:30999:1422278030:16807', sleeping for 30000ms m31101| 2015-01-26T15:01:52.582+0100 [conn225] end connection 192.168.59.3:53003 (6 connections now open) m31101| 2015-01-26T15:01:52.583+0100 [initandlisten] connection accepted from 192.168.59.3:53019 #228 (7 connections now open) m30999| 2015-01-26T15:01:54.237+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c648d2ce5c61cbab9c8350 m30999| 2015-01-26T15:01:54.238+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31100| 2015-01-26T15:01:54.701+0100 [conn247] end connection 192.168.59.3:53005 (10 connections now open) m31100| 2015-01-26T15:01:54.702+0100 [initandlisten] connection accepted from 192.168.59.3:53021 #250 (11 connections now open) m31100| 2015-01-26T15:01:54.869+0100 [conn248] end connection 192.168.59.3:53006 (10 connections now open) m31100| 2015-01-26T15:01:54.870+0100 [initandlisten] connection accepted from 192.168.59.3:53022 #251 (11 connections now open) m30999| 2015-01-26T15:02:00.249+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c648d8ce5c61cbab9c8351 m30999| 2015-01-26T15:02:00.249+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31101| 2015-01-26T15:02:00.956+0100 [conn226] end connection 192.168.1.220:53010 (6 connections now open) m31101| 2015-01-26T15:02:00.956+0100 [initandlisten] connection accepted from 192.168.1.220:53028 #229 (8 connections now open) m30999| 2015-01-26T15:02:06.265+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c648dece5c61cbab9c8352 m30999| 2015-01-26T15:02:06.266+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31102| 2015-01-26T15:02:06.635+0100 [conn223] end connection 192.168.59.3:53011 (4 connections now open) m31102| 2015-01-26T15:02:06.636+0100 [initandlisten] connection accepted from 192.168.59.3:53029 #226 (5 connections now open) m31102| 2015-01-26T15:02:08.728+0100 [conn224] end connection 192.168.59.3:53013 (4 connections now open) m31102| 2015-01-26T15:02:08.729+0100 [initandlisten] connection accepted from 192.168.59.3:53032 #227 (5 connections now open) m31101| 2015-01-26T15:02:08.920+0100 [conn227] end connection 192.168.59.3:53014 (6 connections now open) m31101| 2015-01-26T15:02:08.921+0100 [initandlisten] connection accepted from 192.168.59.3:53033 #230 (7 connections now open) m30999| 2015-01-26T15:02:12.280+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c648e4ce5c61cbab9c8353 m30999| 2015-01-26T15:02:12.281+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31102| 2015-01-26T15:02:15.118+0100 [conn225] end connection 192.168.1.220:53016 (4 connections now open) m31102| 2015-01-26T15:02:15.125+0100 [initandlisten] connection accepted from 192.168.1.220:53034 #228 (5 connections now open) m31100| 2015-01-26T15:02:16.994+0100 [conn249] end connection 192.168.1.220:53017 (10 connections now open) m31100| 2015-01-26T15:02:16.996+0100 [initandlisten] connection accepted from 192.168.1.220:53035 #252 (11 connections now open) m30999| 2015-01-26T15:02:18.292+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c648eace5c61cbab9c8354 m30999| 2015-01-26T15:02:18.293+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m30999| 2015-01-26T15:02:20.769+0100 [LockPinger] cluster Lowang-MacBook-Pro.local:29000 pinged successfully at Mon Jan 26 15:02:20 2015 by distributed lock pinger 'Lowang-MacBook-Pro.local:29000/Lowang-MacBook-Pro.local:30999:1422278030:16807', sleeping for 30000ms m31101| 2015-01-26T15:02:22.701+0100 [conn228] end connection 192.168.59.3:53019 (6 connections now open) m31101| 2015-01-26T15:02:22.702+0100 [initandlisten] connection accepted from 192.168.59.3:53037 #231 (7 connections now open) m30999| 2015-01-26T15:02:24.307+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c648f0ce5c61cbab9c8355 m30999| 2015-01-26T15:02:24.307+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31100| 2015-01-26T15:02:24.830+0100 [conn250] end connection 192.168.59.3:53021 (10 connections now open) m31100| 2015-01-26T15:02:24.830+0100 [initandlisten] connection accepted from 192.168.59.3:53038 #253 (11 connections now open) m31100| 2015-01-26T15:02:24.988+0100 [conn251] end connection 192.168.59.3:53022 (10 connections now open) m31100| 2015-01-26T15:02:24.989+0100 [initandlisten] connection accepted from 192.168.59.3:53039 #254 (11 connections now open) m30999| 2015-01-26T15:02:30.318+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c648f6ce5c61cbab9c8356 m30999| 2015-01-26T15:02:30.319+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31101| 2015-01-26T15:02:31.021+0100 [conn229] end connection 192.168.1.220:53028 (6 connections now open) m31101| 2015-01-26T15:02:31.024+0100 [initandlisten] connection accepted from 192.168.1.220:53043 #232 (7 connections now open) m30999| 2015-01-26T15:02:36.333+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c648fcce5c61cbab9c8357 m30999| 2015-01-26T15:02:36.333+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31102| 2015-01-26T15:02:36.756+0100 [conn226] end connection 192.168.59.3:53029 (4 connections now open) m31102| 2015-01-26T15:02:36.757+0100 [initandlisten] connection accepted from 192.168.59.3:53044 #229 (5 connections now open) m31102| 2015-01-26T15:02:38.852+0100 [conn227] end connection 192.168.59.3:53032 (4 connections now open) m31102| 2015-01-26T15:02:38.853+0100 [initandlisten] connection accepted from 192.168.59.3:53046 #230 (5 connections now open) m31101| 2015-01-26T15:02:39.041+0100 [conn230] end connection 192.168.59.3:53033 (6 connections now open) m31101| 2015-01-26T15:02:39.043+0100 [initandlisten] connection accepted from 192.168.59.3:53047 #233 (7 connections now open) m30999| 2015-01-26T15:02:42.348+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c64902ce5c61cbab9c8358 m30999| 2015-01-26T15:02:42.349+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31102| 2015-01-26T15:02:45.211+0100 [conn228] end connection 192.168.1.220:53034 (4 connections now open) m31102| 2015-01-26T15:02:45.211+0100 [initandlisten] connection accepted from 192.168.1.220:53049 #231 (5 connections now open) m31100| 2015-01-26T15:02:47.080+0100 [conn252] end connection 192.168.1.220:53035 (10 connections now open) m31100| 2015-01-26T15:02:47.083+0100 [initandlisten] connection accepted from 192.168.1.220:53050 #255 (11 connections now open) m30999| 2015-01-26T15:02:48.364+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c64908ce5c61cbab9c8359 m30999| 2015-01-26T15:02:48.364+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m30999| 2015-01-26T15:02:50.775+0100 [LockPinger] cluster Lowang-MacBook-Pro.local:29000 pinged successfully at Mon Jan 26 15:02:50 2015 by distributed lock pinger 'Lowang-MacBook-Pro.local:29000/Lowang-MacBook-Pro.local:30999:1422278030:16807', sleeping for 30000ms m31101| 2015-01-26T15:02:52.815+0100 [conn231] end connection 192.168.59.3:53037 (6 connections now open) m31101| 2015-01-26T15:02:52.816+0100 [initandlisten] connection accepted from 192.168.59.3:53057 #234 (7 connections now open) m31100| 2015-01-26T15:02:54.940+0100 [conn253] end connection 192.168.59.3:53038 (10 connections now open) m31100| 2015-01-26T15:02:54.941+0100 [initandlisten] connection accepted from 192.168.59.3:53059 #256 (11 connections now open) m31100| 2015-01-26T15:02:55.112+0100 [conn254] end connection 192.168.59.3:53039 (10 connections now open) m31100| 2015-01-26T15:02:55.113+0100 [initandlisten] connection accepted from 192.168.59.3:53060 #257 (11 connections now open) 2015-01-26T15:02:56.389+0100 [ReplicaSetMonitorWatcher] Socket recv() timeout 192.168.59.103:27017 2015-01-26T15:02:56.389+0100 [ReplicaSetMonitorWatcher] SocketException: remote: 192.168.59.103:27017 error: 9001 socket exception [RECV_TIMEOUT] server [192.168.59.103:27017] 2015-01-26T15:02:56.389+0100 [ReplicaSetMonitorWatcher] DBClientCursor::init call() failed 2015-01-26T15:02:56.391+0100 [ReplicaSetMonitorWatcher] Detected bad connection created at 1422279930562702 microSec, clearing pool for 192.168.59.103:27017 of 0 connections 2015-01-26T15:02:56.391+0100 [ReplicaSetMonitorWatcher] warning: No primary detected for set test-rs0 m30999| 2015-01-26T15:02:57.143+0100 [ReplicaSetMonitorWatcher] Socket recv() timeout 192.168.59.103:27017 m30999| 2015-01-26T15:02:57.143+0100 [ReplicaSetMonitorWatcher] SocketException: remote: 192.168.59.103:27017 error: 9001 socket exception [RECV_TIMEOUT] server [192.168.59.103:27017] m30999| 2015-01-26T15:02:57.143+0100 [ReplicaSetMonitorWatcher] DBClientCursor::init call() failed m30999| 2015-01-26T15:02:57.145+0100 [ReplicaSetMonitorWatcher] Detected bad connection created at 1422279931255298 microSec, clearing pool for 192.168.59.103:27017 of 0 connections m30999| 2015-01-26T15:02:57.145+0100 [ReplicaSetMonitorWatcher] warning: No primary detected for set test-rs0 m31100| 2015-01-26T15:03:00.237+0100 [rsHealthPoll] DBClientCursor::init call() failed m31100| 2015-01-26T15:03:00.239+0100 [rsHealthPoll] replSet info 192.168.59.103:27017 is down (or slow to respond): m31100| 2015-01-26T15:03:00.239+0100 [rsHealthPoll] replSet member 192.168.59.103:27017 is now in state DOWN m31100| 2015-01-26T15:03:00.239+0100 [rsMgr] not electing self, Lowang-MacBook-Pro.local:31102 would veto with 'Lowang-MacBook-Pro.local:31100 is trying to elect itself but 192.168.59.103:27017 is already primary and more up-to-date' m31101| 2015-01-26T15:03:00.311+0100 [rsHealthPoll] DBClientCursor::init call() failed m31101| 2015-01-26T15:03:00.311+0100 [rsHealthPoll] replSet info 192.168.59.103:27017 is down (or slow to respond): m31101| 2015-01-26T15:03:00.311+0100 [rsHealthPoll] replSet member 192.168.59.103:27017 is now in state DOWN m31101| 2015-01-26T15:03:00.311+0100 [rsMgr] not electing self, Lowang-MacBook-Pro.local:31102 would veto with 'Lowang-MacBook-Pro.local:31101 is trying to elect itself but 192.168.59.103:27017 is already primary and more up-to-date' m31102| 2015-01-26T15:03:00.324+0100 [rsHealthPoll] DBClientCursor::init call() failed m31102| 2015-01-26T15:03:00.324+0100 [rsHealthPoll] replSet info 192.168.59.103:27017 is down (or slow to respond): m31102| 2015-01-26T15:03:00.324+0100 [rsHealthPoll] replSet member 192.168.59.103:27017 is now in state DOWN m31102| 2015-01-26T15:03:06.863+0100 [conn229] end connection 192.168.59.3:53044 (4 connections now open) m31100| 2015-01-26T15:03:07.255+0100 [rsHealthPoll] replset info 192.168.59.103:27017 heartbeat failed, retrying m31101| 2015-01-26T15:03:07.255+0100 [rsMgr] replSet info electSelf 1 m31100| 2015-01-26T15:03:07.255+0100 [conn256] replSet info voting yea for Lowang-MacBook-Pro.local:31101 (1) m31102| 2015-01-26T15:03:07.255+0100 [conn230] replSet info voting yea for Lowang-MacBook-Pro.local:31101 (1) m31101| 2015-01-26T15:03:07.324+0100 [rsHealthPoll] replset info 192.168.59.103:27017 heartbeat failed, retrying m31102| 2015-01-26T15:03:07.343+0100 [rsHealthPoll] replset info 192.168.59.103:27017 heartbeat failed, retrying m31101| 2015-01-26T15:03:08.836+0100 [rsMgr] replSet PRIMARY m31100| 2015-01-26T15:03:08.836+0100 [conn171] end connection 192.168.59.3:52488 (10 connections now open) m31100| 2015-01-26T15:03:08.871+0100 [rsHealthPoll] replSet member Lowang-MacBook-Pro.local:31101 is now in state PRIMARY m31102| 2015-01-26T15:03:08.986+0100 [conn230] end connection 192.168.59.3:53046 (3 connections now open) m31101| 2015-01-26T15:03:09.168+0100 [conn233] end connection 192.168.59.3:53047 (6 connections now open) m31100| 2015-01-26T15:12:12.980+0100 [conn255] end connection 192.168.1.220:53050 (9 connections now open) m31100| 2015-01-26T15:12:12.982+0100 [conn173] end connection 192.168.1.220:52490 (8 connections now open) m31102| 2015-01-26T15:12:12.981+0100 [conn231] end connection 192.168.1.220:53049 (2 connections now open) m31101| 2015-01-26T15:12:12.981+0100 [conn232] end connection 192.168.1.220:53043 (5 connections now open) m29000| 2015-01-26T15:12:12.983+0100 [conn10] end connection 192.168.1.220:52535 (8 connections now open) m29000| 2015-01-26T15:12:12.983+0100 [conn9] end connection 192.168.1.220:52534 (7 connections now open) m29000| 2015-01-26T15:12:12.983+0100 [conn8] end connection 192.168.1.220:52533 (6 connections now open) m31100| 2015-01-26T15:12:14.469+0100 [rsHealthPoll] getaddrinfo("Lowang-MacBook-Pro.local") failed: nodename nor servname provided, or not known m31100| 2015-01-26T15:12:14.469+0100 [rsHealthPoll] couldn't connect to Lowang-MacBook-Pro.local:31102: couldn't connect to server Lowang-MacBook-Pro.local:31102 (0.0.0.0), address resolved to 0.0.0.0 m31102| 2015-01-26T15:12:14.903+0100 [initandlisten] connection accepted from 192.168.59.3:53086 #232 (3 connections now open) m31101| 2015-01-26T15:12:14.903+0100 [initandlisten] connection accepted from 192.168.59.3:53087 #235 (6 connections now open) m31102| 2015-01-26T15:12:14.904+0100 [rsHealthPoll] replSet member Lowang-MacBook-Pro.local:31101 is now in state PRIMARY m31102| 2015-01-26T15:12:14.904+0100 [initandlisten] connection accepted from 192.168.59.3:53088 #233 (4 connections now open) m31100| 2015-01-26T15:12:14.905+0100 [rsMgr] not electing self, Lowang-MacBook-Pro.local:31102 would veto with 'Lowang-MacBook-Pro.local:31100 is trying to elect itself but Lowang-MacBook-Pro.local:31101 is already primary and more up-to-date' m31100| 2015-01-26T15:12:16.792+0100 [rsHealthPoll] replSet member 192.168.59.103:27017 is up m31100| 2015-01-26T15:12:16.792+0100 [rsHealthPoll] replSet member 192.168.59.103:27017 is now in state PRIMARY m31100| 2015-01-26T15:12:16.792+0100 [rsMgr] replSet info two primaries (transiently) m31101| 2015-01-26T15:12:16.861+0100 [rsHealthPoll] replSet member 192.168.59.103:27017 is up m31101| 2015-01-26T15:12:16.861+0100 [rsHealthPoll] replSet member 192.168.59.103:27017 is now in state PRIMARY m31101| 2015-01-26T15:12:16.862+0100 [rsMgr] another PRIMARY detected but it should step down since it was elected earlier than me m31101| 2015-01-26T15:12:16.862+0100 [rsMgr] another PRIMARY detected but it should step down since it was elected earlier than me m31102| 2015-01-26T15:12:16.883+0100 [rsHealthPoll] replSet member 192.168.59.103:27017 is up m31102| 2015-01-26T15:12:16.883+0100 [rsHealthPoll] replSet member 192.168.59.103:27017 is now in state PRIMARY m31102| 2015-01-26T15:12:16.883+0100 [rsMgr] replSet info two primaries (transiently) m30999| 2015-01-26T15:12:17.857+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c64b41ce5c61cbab9c835a m30999| 2015-01-26T15:12:17.858+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31101| 2015-01-26T15:12:20.466+0100 [initandlisten] connection accepted from 192.168.1.220:53107 #236 (7 connections now open) m31100| 2015-01-26T15:12:20.468+0100 [rsBackgroundSync] replSet sync source problem: 10278 dbclient error communicating with server: 192.168.59.103:27017 m31100| 2015-01-26T15:12:20.468+0100 [rsBackgroundSync] replSet syncing to: Lowang-MacBook-Pro.local:31101 m31101| 2015-01-26T15:12:20.468+0100 [conn236] end connection 192.168.1.220:53107 (6 connections now open) m30999| 2015-01-26T15:12:20.468+0100 [WriteBackListener-192.168.59.103:27017] DBClientCursor::init call() failed m30999| 2015-01-26T15:12:20.468+0100 [WriteBackListener-192.168.59.103:27017] Detected bad connection created at 1422280260800723 microSec, clearing pool for 192.168.59.103:27017 of 0 connections m30999| 2015-01-26T15:12:20.468+0100 [WriteBackListener-192.168.59.103:27017] WriteBackListener exception : DBClientBase::findN: transport error: 192.168.59.103:27017 ns: admin.$cmd query: { writebacklisten: ObjectId('54c63d8ece5c61cbab9c816b') } m31101| 2015-01-26T15:12:20.469+0100 [initandlisten] connection accepted from 192.168.59.3:53108 #237 (7 connections now open) m31100| 2015-01-26T15:12:20.469+0100 [rsBackgroundSync] replset setting syncSourceFeedback to Lowang-MacBook-Pro.local:31101 m31101| 2015-01-26T15:12:20.471+0100 [initandlisten] connection accepted from 192.168.59.3:53109 #238 (8 connections now open) m31100| 2015-01-26T15:12:20.476+0100 [initandlisten] connection accepted from 192.168.1.220:53110 #258 (9 connections now open) m31101| 2015-01-26T15:12:20.498+0100 [initandlisten] connection accepted from 192.168.1.220:53111 #239 (9 connections now open) m31101| 2015-01-26T15:12:20.507+0100 [initandlisten] connection accepted from 192.168.1.220:53112 #240 (10 connections now open) m31102| 2015-01-26T15:12:20.606+0100 [initandlisten] connection accepted from 192.168.1.220:53113 #234 (5 connections now open) m31100| 2015-01-26T15:12:20.807+0100 [rsHealthPoll] replSet member 192.168.59.103:27017 is now in state SECONDARY m31101| 2015-01-26T15:12:20.871+0100 [rsHealthPoll] replSet member 192.168.59.103:27017 is now in state SECONDARY m31102| 2015-01-26T15:12:20.896+0100 [rsHealthPoll] replSet member 192.168.59.103:27017 is now in state SECONDARY m31101| 2015-01-26T15:12:22.472+0100 [initandlisten] connection accepted from 192.168.1.220:53116 #241 (11 connections now open) m30999| 2015-01-26T15:12:23.301+0100 [LockPinger] warning: Lock pinger for addr: Lowang-MacBook-Pro.local:29000, proc: Lowang-MacBook-Pro.local:30999:1422278030:16807 was inactive for 572527 ms m30999| 2015-01-26T15:12:23.302+0100 [LockPinger] cluster Lowang-MacBook-Pro.local:29000 pinged successfully at Mon Jan 26 15:12:23 2015 by distributed lock pinger 'Lowang-MacBook-Pro.local:29000/Lowang-MacBook-Pro.local:30999:1422278030:16807', sleeping for 30000ms m30999| 2015-01-26T15:12:23.864+0100 [Balancer] Socket closed remotely, no longer connected (idle 569 secs, remote host 192.168.59.103:27017) m31101| 2015-01-26T15:12:23.866+0100 [initandlisten] connection accepted from 192.168.59.3:53120 #242 (12 connections now open) m30999| 2015-01-26T15:12:23.867+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c64b47ce5c61cbab9c835b m30999| 2015-01-26T15:12:23.867+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31101| 2015-01-26T15:12:25.415+0100 [conn234] end connection 192.168.59.3:53057 (11 connections now open) m31101| 2015-01-26T15:12:25.417+0100 [initandlisten] connection accepted from 192.168.59.3:53124 #243 (12 connections now open) m31101| 2015-01-26T15:12:25.969+0100 [conn241] end connection 192.168.1.220:53116 (11 connections now open) m31102| 2015-01-26T15:12:25.971+0100 [conn234] end connection 192.168.1.220:53113 (4 connections now open) m31101| 2015-01-26T15:12:25.971+0100 [conn240] end connection 192.168.1.220:53112 (10 connections now open) m31100| 2015-01-26T15:12:25.983+0100 [conn258] end connection 192.168.1.220:53110 (8 connections now open) m31100| 2015-01-26T15:12:27.542+0100 [conn256] end connection 192.168.59.3:53059 (7 connections now open) m31100| 2015-01-26T15:12:27.543+0100 [initandlisten] connection accepted from 192.168.59.3:53135 #259 (8 connections now open) m31100| 2015-01-26T15:12:27.732+0100 [conn257] end connection 192.168.59.3:53060 (7 connections now open) m31100| 2015-01-26T15:12:27.733+0100 [initandlisten] connection accepted from 192.168.59.3:53137 #260 (8 connections now open) m30999| 2015-01-26T15:12:29.875+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c64b4dce5c61cbab9c835c m30999| 2015-01-26T15:12:29.875+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31101| 2015-01-26T15:12:30.600+0100 [conn239] end connection 192.168.1.220:53111 (9 connections now open) m30999| 2015-01-26T15:12:32.329+0100 [mongosMain] connection accepted from 127.0.0.1:53142 #5 (3 connections now open) m30999| 2015-01-26T15:12:32.330+0100 [conn5] end connection 127.0.0.1:53142 (2 connections now open) m30999| 2015-01-26T15:12:32.331+0100 [mongosMain] connection accepted from 127.0.0.1:53143 #6 (3 connections now open) m30999| 2015-01-26T15:12:32.331+0100 [conn6] Socket closed remotely, no longer connected (idle 582 secs, remote host 192.168.59.103:27017) m31101| 2015-01-26T15:12:32.332+0100 [initandlisten] connection accepted from 192.168.59.3:53144 #244 (10 connections now open) m31101| 2015-01-26T15:12:32.334+0100 [conn244] first cluster operation detected, adding sharding hook to enable versioning and authentication to remote servers m31101| 2015-01-26T15:12:32.339+0100 [conn244] remote client 192.168.59.3:53144 initialized this host (test-rs0/192.168.59.103:27017,Lowang-MacBook-Pro.local:31100,Lowang-MacBook-Pro.local:31101) as shard test-rs0 m31101| 2015-01-26T15:12:32.339+0100 [conn244] remotely refreshing metadata for mongodb_outage_test.data with requested shard version 1|16||54c64644ce5c61cbab9c82e2, current shard version is 0|0||000000000000000000000000, current metadata version is 0|0||000000000000000000000000 m29000| 2015-01-26T15:12:32.342+0100 [initandlisten] connection accepted from 192.168.59.3:53145 #11 (7 connections now open) m31101| 2015-01-26T15:12:32.344+0100 [conn244] collection mongodb_outage_test.data was previously unsharded, new metadata loaded with shard version 1|16||54c64644ce5c61cbab9c82e2 m31101| 2015-01-26T15:12:32.344+0100 [conn244] collection version was loaded at version 1|16||54c64644ce5c61cbab9c82e2, took 3ms m31101| 2015-01-26T15:12:33.002+0100 [initandlisten] connection accepted from 192.168.1.220:53146 #245 (11 connections now open) m31100| 2015-01-26T15:12:33.013+0100 [initandlisten] connection accepted from 192.168.1.220:53147 #261 (9 connections now open) m31102| 2015-01-26T15:12:33.146+0100 [initandlisten] connection accepted from 192.168.1.220:53148 #235 (5 connections now open) m31100| 2015-01-26T15:12:34.880+0100 [clientcursormon] mem (MB) res:178 virt:3290 m31100| 2015-01-26T15:12:34.880+0100 [clientcursormon] mapped (incl journal view):768 m31100| 2015-01-26T15:12:34.881+0100 [clientcursormon] connections:9 m31100| 2015-01-26T15:12:34.881+0100 [clientcursormon] replication threads:32 m31101| 2015-01-26T15:12:34.896+0100 [clientcursormon] mem (MB) res:178 virt:3289 m31101| 2015-01-26T15:12:34.896+0100 [clientcursormon] mapped (incl journal view):768 m31101| 2015-01-26T15:12:34.896+0100 [clientcursormon] connections:11 m31101| 2015-01-26T15:12:34.896+0100 [clientcursormon] replication threads:32 m31102| 2015-01-26T15:12:35.089+0100 [clientcursormon] mem (MB) res:34 virt:2573 m31102| 2015-01-26T15:12:35.089+0100 [clientcursormon] mapped (incl journal view):64 m31102| 2015-01-26T15:12:35.089+0100 [clientcursormon] connections:5 m31102| 2015-01-26T15:12:35.089+0100 [clientcursormon] replication threads:32 m31100| 2015-01-26T15:12:35.435+0100 [rsMgr] stepping down Lowang-MacBook-Pro.local:31101 (priority 1), 192.168.59.103:27017 is priority 2 and 0 seconds behind m31101| 2015-01-26T15:12:35.435+0100 [conn243] 192.168.59.103:27017 is now closest at 54c64738:a2 m31101| 2015-01-26T15:12:35.436+0100 [conn243] replSet info stepping down as primary secs=1 m31101| 2015-01-26T15:12:35.436+0100 [conn243] replSet relinquishing primary state m31101| 2015-01-26T15:12:35.436+0100 [conn243] replSet SECONDARY m31101| 2015-01-26T15:12:35.436+0100 [conn243] replSet closing client sockets after relinquishing primary m31101| 2015-01-26T15:12:35.436+0100 [conn7] end connection 192.168.59.3:51573 (10 connections now open) m31101| 2015-01-26T15:12:35.436+0100 [conn1] end connection 127.0.0.1:51533 (10 connections now open) m31101| 2015-01-26T15:12:35.436+0100 [conn238] end connection 192.168.59.3:53109 (10 connections now open) m31101| 2015-01-26T15:12:35.436+0100 [conn8] end connection 192.168.59.3:51578 (10 connections now open) m31101| 2015-01-26T15:12:35.436+0100 [conn244] end connection 192.168.59.3:53144 (10 connections now open) m31101| 2015-01-26T15:12:35.436+0100 [conn242] end connection 192.168.59.3:53120 (10 connections now open) m31100| 2015-01-26T15:12:35.436+0100 [conn172] end connection 192.168.59.3:52489 (8 connections now open) m30999| 2015-01-26T15:12:35.436+0100 [WriteBackListener-Lowang-MacBook-Pro.local:31101] DBClientCursor::init call() failed m30999| 2015-01-26T15:12:35.436+0100 [WriteBackListener-Lowang-MacBook-Pro.local:31101] Detected bad connection created at 1422278103958099 microSec, clearing pool for Lowang-MacBook-Pro.local:31101 of 0 connections m30999| 2015-01-26T15:12:35.436+0100 [WriteBackListener-Lowang-MacBook-Pro.local:31101] WriteBackListener exception : DBClientBase::findN: transport error: Lowang-MacBook-Pro.local:31101 ns: admin.$cmd query: { writebacklisten: ObjectId('54c63d8ece5c61cbab9c816b') } m31100| 2015-01-26T15:12:35.436+0100 [rsBackgroundSync] replSet sync source problem: 10278 dbclient error communicating with server: Lowang-MacBook-Pro.local:31101 m29000| 2015-01-26T15:12:35.437+0100 [conn11] end connection 192.168.59.3:53145 (6 connections now open) m31100| 2015-01-26T15:12:35.437+0100 [rsBackgroundSync] replSet syncing to: Lowang-MacBook-Pro.local:31101 m31101| 2015-01-26T15:12:35.438+0100 [initandlisten] connection accepted from 192.168.59.3:53149 #246 (6 connections now open) m31100| 2015-01-26T15:12:35.438+0100 [rsBackgroundSync] replset setting syncSourceFeedback to Lowang-MacBook-Pro.local:31101 m31101| 2015-01-26T15:12:35.439+0100 [initandlisten] connection accepted from 192.168.59.3:53150 #247 (7 connections now open) m31101| 2015-01-26T15:12:35.572+0100 [conn237] SocketException handling request, closing client connection: 9001 socket exception [SEND_ERROR] server [192.168.59.3:53108] m31102| 2015-01-26T15:12:35.749+0100 [rsMgr] stepping down Lowang-MacBook-Pro.local:31101 (priority 1), 192.168.59.103:27017 is priority 2 and 0 seconds behind m31102| 2015-01-26T15:12:35.749+0100 [rsMgr] stepping down Lowang-MacBook-Pro.local:31101 failed: { ok: 0.0, errmsg: "not primary so can't step down" } m30999| 2015-01-26T15:12:35.880+0100 [Balancer] Socket closed remotely, no longer connected (idle 6 secs, remote host 192.168.59.3:31101) m31101| 2015-01-26T15:12:35.882+0100 [initandlisten] connection accepted from 192.168.59.3:53151 #248 (7 connections now open) m30999| 2015-01-26T15:12:35.884+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c64b53ce5c61cbab9c835d m30999| 2015-01-26T15:12:35.885+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31101| 2015-01-26T15:12:36.441+0100 [initandlisten] connection accepted from 192.168.59.3:53152 #249 (8 connections now open) m31102| 2015-01-26T15:12:36.946+0100 [rsHealthPoll] replSet member Lowang-MacBook-Pro.local:31101 is now in state SECONDARY m31102| 2015-01-26T15:12:36.946+0100 [rsMgr] stepping down Lowang-MacBook-Pro.local:31101 (priority 1), 192.168.59.103:27017 is priority 2 and 0 seconds behind m31102| 2015-01-26T15:12:36.946+0100 [rsMgr] stepping down Lowang-MacBook-Pro.local:31101 failed: { ok: 0.0, errmsg: "not primary so can't step down" } m30999| 2015-01-26T15:12:37.386+0100 [conn6] Socket closed remotely, no longer connected (idle 5 secs, remote host 192.168.59.3:31101) m31100| 2015-01-26T15:12:37.387+0100 [conn194] end connection 192.168.59.3:52708 (7 connections now open) m31100| 2015-01-26T15:12:37.439+0100 [rsHealthPoll] replSet member Lowang-MacBook-Pro.local:31101 is now in state SECONDARY m31100| 2015-01-26T15:12:37.440+0100 [rsMgr] stepping down Lowang-MacBook-Pro.local:31101 (priority 1), 192.168.59.103:27017 is priority 2 and 0 seconds behind m31100| 2015-01-26T15:12:37.440+0100 [rsMgr] stepping down Lowang-MacBook-Pro.local:31101 failed: { ok: 0.0, errmsg: "not primary so can't step down" } m31100| 2015-01-26T15:12:37.441+0100 [rsMgr] not electing self, 192.168.59.103:27017 would veto with 'Lowang-MacBook-Pro.local:31100 has lower priority than 192.168.59.103:27017' m31101| 2015-01-26T15:12:37.893+0100 [initandlisten] connection accepted from 192.168.59.3:53153 #250 (9 connections now open) m31101| 2015-01-26T15:12:37.894+0100 [conn250] remote client 192.168.59.3:53153 initialized this host (test-rs0/192.168.59.103:27017,Lowang-MacBook-Pro.local:31100,Lowang-MacBook-Pro.local:31101) as shard test-rs0 m30999| 2015-01-26T15:12:37.894+0100 [conn6] got not master for: Lowang-MacBook-Pro.local:31101 m30999| 2015-01-26T15:12:37.894+0100 [conn6] DBClientCursor::init call() failed m31101| 2015-01-26T15:12:37.894+0100 [conn250] end connection 192.168.59.3:53153 (8 connections now open) m30999| 2015-01-26T15:12:37.894+0100 [conn6] Detected bad connection created at 1422278030422300 microSec, clearing pool for Lowang-MacBook-Pro.local:31101 of 0 connections m30999| 2015-01-26T15:12:37.897+0100 [conn6] warning: No primary detected for set test-rs0 m31101| 2015-01-26T15:12:38.407+0100 [initandlisten] connection accepted from 192.168.59.3:53155 #251 (9 connections now open) m30999| 2015-01-26T15:12:38.409+0100 [conn6] warning: No primary detected for set test-rs0 m31101| 2015-01-26T15:12:38.410+0100 [initandlisten] connection accepted from 192.168.59.3:53156 #252 (10 connections now open) m31101| 2015-01-26T15:12:38.410+0100 [conn252] end connection 192.168.59.3:53156 (9 connections now open) m30999| 2015-01-26T15:12:38.917+0100 [conn6] warning: No primary detected for set test-rs0 m31101| 2015-01-26T15:12:38.919+0100 [initandlisten] connection accepted from 192.168.59.3:53157 #253 (10 connections now open) m31101| 2015-01-26T15:12:38.919+0100 [conn253] end connection 192.168.59.3:53157 (9 connections now open) m31101| 2015-01-26T15:12:39.018+0100 [conn245] end connection 192.168.1.220:53146 (8 connections now open) m31101| 2015-01-26T15:12:39.023+0100 [initandlisten] connection accepted from 192.168.1.220:53158 #254 (9 connections now open) m30999| 2015-01-26T15:12:39.426+0100 [conn6] warning: No primary detected for set test-rs0 m30999| 2015-01-26T15:12:39.943+0100 [conn6] warning: No primary detected for set test-rs0 m31100| 2015-01-26T15:12:39.944+0100 [initandlisten] connection accepted from 192.168.59.3:53161 #262 (8 connections now open) m31100| 2015-01-26T15:12:39.944+0100 [conn262] end connection 192.168.59.3:53161 (7 connections now open) m30999| 2015-01-26T15:12:40.448+0100 [conn6] warning: No primary detected for set test-rs0 m31101| 2015-01-26T15:12:40.942+0100 [rsMgr] not electing self, 192.168.59.103:27017 would veto with 'Lowang-MacBook-Pro.local:31101 has lower priority than 192.168.59.103:27017' m30999| 2015-01-26T15:12:40.962+0100 [conn6] warning: No primary detected for set test-rs0 m31101| 2015-01-26T15:12:40.963+0100 [initandlisten] connection accepted from 192.168.59.3:53164 #255 (10 connections now open) m31101| 2015-01-26T15:12:40.963+0100 [conn255] end connection 192.168.59.3:53164 (9 connections now open) m30999| 2015-01-26T15:12:41.473+0100 [conn6] warning: No primary detected for set test-rs0 m31100| 2015-01-26T15:12:41.474+0100 [initandlisten] connection accepted from 192.168.59.3:53165 #263 (8 connections now open) m31100| 2015-01-26T15:12:41.474+0100 [conn263] end connection 192.168.59.3:53165 (7 connections now open) m30999| 2015-01-26T15:12:41.892+0100 [Balancer] warning: No primary detected for set test-rs0 m30999| 2015-01-26T15:12:41.894+0100 [Balancer] scoped connection to test-rs0/192.168.59.103:27017,Lowang-MacBook-Pro.local:31100,Lowang-MacBook-Pro.local:31101 not being returned to the pool m30999| 2015-01-26T15:12:41.894+0100 [Balancer] scoped connection to Lowang-MacBook-Pro.local:29000 not being returned to the pool m31101| 2015-01-26T15:12:41.894+0100 [conn248] end connection 192.168.59.3:53151 (8 connections now open) m30999| 2015-01-26T15:12:41.894+0100 [Balancer] caught exception while doing balance: ReplicaSetMonitor no master found for set: test-rs0 m29000| 2015-01-26T15:12:41.895+0100 [conn7] end connection 192.168.59.3:51571 (5 connections now open) m30999| 2015-01-26T15:12:41.977+0100 [conn6] warning: No primary detected for set test-rs0 m31101| 2015-01-26T15:12:41.978+0100 [initandlisten] connection accepted from 192.168.59.3:53166 #256 (9 connections now open) m31101| 2015-01-26T15:12:41.979+0100 [conn256] end connection 192.168.59.3:53166 (8 connections now open) m30999| 2015-01-26T15:12:42.486+0100 [conn6] warning: No primary detected for set test-rs0 m31101| 2015-01-26T15:12:42.487+0100 [initandlisten] connection accepted from 192.168.59.3:53167 #257 (9 connections now open) m31101| 2015-01-26T15:12:42.488+0100 [conn257] end connection 192.168.59.3:53167 (8 connections now open) m31100| 2015-01-26T15:12:42.875+0100 [rsMgr] not electing self, 192.168.59.103:27017 would veto with 'Lowang-MacBook-Pro.local:31100 has lower priority than 192.168.59.103:27017' m30999| 2015-01-26T15:12:42.993+0100 [conn6] warning: No primary detected for set test-rs0 m31100| 2015-01-26T15:12:42.994+0100 [initandlisten] connection accepted from 192.168.59.3:53168 #264 (8 connections now open) m31100| 2015-01-26T15:12:42.994+0100 [conn264] end connection 192.168.59.3:53168 (7 connections now open) m30999| 2015-01-26T15:12:43.502+0100 [conn6] warning: No primary detected for set test-rs0 2015-01-26T15:12:43.948+0100 [ReplicaSetMonitorWatcher] Socket closed remotely, no longer connected (idle 10 secs, remote host 192.168.59.3:31101) m31101| 2015-01-26T15:12:43.950+0100 [initandlisten] connection accepted from 192.168.59.3:53171 #258 (9 connections now open) 2015-01-26T15:12:43.953+0100 [ReplicaSetMonitorWatcher] warning: No primary detected for set test-rs0 m31102| 2015-01-26T15:12:43.966+0100 [conn235] replSet info voting yea for 192.168.59.103:27017 (3) m31101| 2015-01-26T15:12:43.966+0100 [conn254] replSet info voting yea for 192.168.59.103:27017 (3) m31100| 2015-01-26T15:12:43.966+0100 [conn261] replSet info voting yea for 192.168.59.103:27017 (3) m30999| 2015-01-26T15:12:44.016+0100 [conn6] warning: No primary detected for set test-rs0 m31101| 2015-01-26T15:12:44.018+0100 [initandlisten] connection accepted from 192.168.59.3:53172 #259 (10 connections now open) m31101| 2015-01-26T15:12:44.018+0100 [conn259] end connection 192.168.59.3:53172 (9 connections now open) m30999| 2015-01-26T15:12:44.528+0100 [conn6] warning: No primary detected for set test-rs0 m31101| 2015-01-26T15:12:44.529+0100 [initandlisten] connection accepted from 192.168.59.3:53176 #260 (10 connections now open) m31101| 2015-01-26T15:12:44.530+0100 [conn260] end connection 192.168.59.3:53176 (9 connections now open) m30999| 2015-01-26T15:12:44.705+0100 [ReplicaSetMonitorWatcher] warning: No primary detected for set test-rs0 m31101| 2015-01-26T15:12:44.957+0100 [conn235] end connection 192.168.59.3:53087 (8 connections now open) m31102| 2015-01-26T15:12:44.957+0100 [conn232] end connection 192.168.59.3:53086 (4 connections now open) m31102| 2015-01-26T15:12:44.957+0100 [conn233] end connection 192.168.59.3:53088 (3 connections now open) m31102| 2015-01-26T15:12:44.958+0100 [initandlisten] connection accepted from 192.168.59.3:53177 #236 (4 connections now open) m31102| 2015-01-26T15:12:44.958+0100 [initandlisten] connection accepted from 192.168.59.3:53178 #237 (5 connections now open) m31101| 2015-01-26T15:12:44.958+0100 [initandlisten] connection accepted from 192.168.59.3:53179 #261 (9 connections now open) m30999| 2015-01-26T15:12:45.035+0100 [conn6] warning: No primary detected for set test-rs0 m31100| 2015-01-26T15:12:45.037+0100 [initandlisten] connection accepted from 192.168.59.3:53180 #265 (8 connections now open) m31100| 2015-01-26T15:12:45.037+0100 [conn265] end connection 192.168.59.3:53180 (7 connections now open) m30999| 2015-01-26T15:12:45.547+0100 [conn6] warning: No primary detected for set test-rs0 m30999| 2015-01-26T15:12:45.547+0100 [conn6] warning: Primary for test-rs0/192.168.59.103:27017,Lowang-MacBook-Pro.local:31100,Lowang-MacBook-Pro.local:31101 was down before, bypassing setShardVersion. The local replica set view and targeting may be stale. m31100| 2015-01-26T15:12:45.548+0100 [initandlisten] connection accepted from 192.168.59.3:53181 #266 (8 connections now open) m31100| 2015-01-26T15:12:45.548+0100 [conn266] end connection 192.168.59.3:53181 (7 connections now open) m31101| 2015-01-26T15:12:45.578+0100 [rsMgr] not electing self, 192.168.59.103:27017 would veto with 'Lowang-MacBook-Pro.local:31101 has lower priority than 192.168.59.103:27017' m30999| 2015-01-26T15:12:47.055+0100 [mongosMain] connection accepted from 127.0.0.1:53188 #7 (4 connections now open) m29000| 2015-01-26T15:12:47.899+0100 [initandlisten] connection accepted from 192.168.59.3:53190 #12 (6 connections now open) m30999| 2015-01-26T15:12:48.317+0100 [mongosMain] connection accepted from 127.0.0.1:53194 #8 (5 connections now open) m30999| 2015-01-26T15:12:50.080+0100 [mongosMain] connection accepted from 127.0.0.1:53195 #9 (6 connections now open) m30999| 2015-01-26T15:12:51.053+0100 [conn6] Socket recv() timeout 192.168.59.103:27017 m30999| 2015-01-26T15:12:51.053+0100 [conn6] SocketException: remote: 192.168.59.103:27017 error: 9001 socket exception [RECV_TIMEOUT] server [192.168.59.103:27017] m30999| 2015-01-26T15:12:51.053+0100 [conn6] DBClientCursor::init call() failed m30999| 2015-01-26T15:12:51.053+0100 [conn6] Detected bad connection created at 1422281544692589 microSec, clearing pool for 192.168.59.103:27017 of 0 connections m30999| 2015-01-26T15:12:51.053+0100 [conn6] warning: No primary detected for set test-rs0 m30999| 2015-01-26T15:12:51.053+0100 [conn6] warning: failed to initialize new replica set connection version, will initialize on first use m31100| 2015-01-26T15:12:51.054+0100 [initandlisten] connection accepted from 192.168.59.3:53197 #267 (8 connections now open) m31100| 2015-01-26T15:12:51.055+0100 [conn267] end connection 192.168.59.3:53197 (7 connections now open) m30999| 2015-01-26T15:12:51.055+0100 [conn6] end connection 127.0.0.1:53143 (5 connections now open) m30999| 2015-01-26T15:12:51.336+0100 [mongosMain] connection accepted from 127.0.0.1:53198 #10 (6 connections now open) m30999| 2015-01-26T15:12:53.099+0100 [mongosMain] connection accepted from 127.0.0.1:53199 #11 (7 connections now open) m30999| 2015-01-26T15:12:53.119+0100 [PeriodicTaskRunner] Socket closed remotely, no longer connected (idle 603 secs, remote host 192.168.59.103:27017) m30999| 2015-01-26T15:12:53.305+0100 [LockPinger] cluster Lowang-MacBook-Pro.local:29000 pinged successfully at Mon Jan 26 15:12:53 2015 by distributed lock pinger 'Lowang-MacBook-Pro.local:29000/Lowang-MacBook-Pro.local:30999:1422278030:16807', sleeping for 30000ms m30999| 2015-01-26T15:12:54.357+0100 [mongosMain] connection accepted from 127.0.0.1:53200 #12 (8 connections now open) m29000| 2015-01-26T15:12:55.467+0100 [clientcursormon] mem (MB) res:50 virt:2610 m29000| 2015-01-26T15:12:55.467+0100 [clientcursormon] mapped (incl journal view):128 m29000| 2015-01-26T15:12:55.467+0100 [clientcursormon] connections:6 m31101| 2015-01-26T15:12:55.473+0100 [conn243] end connection 192.168.59.3:53124 (8 connections now open) m31101| 2015-01-26T15:12:55.474+0100 [initandlisten] connection accepted from 192.168.59.3:53203 #262 (9 connections now open) m30999| 2015-01-26T15:12:56.054+0100 [conn8] warning: No primary detected for set test-rs0 m30999| 2015-01-26T15:12:56.055+0100 [Balancer] scoped connection to Lowang-MacBook-Pro.local:29000 not being returned to the pool m30999| 2015-01-26T15:12:56.055+0100 [Balancer] caught exception while doing balance: ReplicaSetMonitor no master found for set: test-rs0 m29000| 2015-01-26T15:12:56.055+0100 [conn6] end connection 192.168.59.3:51570 (5 connections now open) m30999| 2015-01-26T15:12:56.119+0100 [mongosMain] connection accepted from 127.0.0.1:53205 #13 (9 connections now open) m31100| 2015-01-26T15:12:56.886+0100 [rsHealthPoll] DBClientCursor::init call() failed m31100| 2015-01-26T15:12:56.886+0100 [rsHealthPoll] replSet info 192.168.59.103:27017 is down (or slow to respond): m31100| 2015-01-26T15:12:56.886+0100 [rsHealthPoll] replSet member 192.168.59.103:27017 is now in state DOWN m31101| 2015-01-26T15:12:56.952+0100 [rsHealthPoll] DBClientCursor::init call() failed m31101| 2015-01-26T15:12:56.952+0100 [rsHealthPoll] replSet info 192.168.59.103:27017 is down (or slow to respond): m31101| 2015-01-26T15:12:56.952+0100 [rsHealthPoll] replSet member 192.168.59.103:27017 is now in state DOWN m31102| 2015-01-26T15:12:56.966+0100 [rsHealthPoll] DBClientCursor::init call() failed m31102| 2015-01-26T15:12:56.966+0100 [rsHealthPoll] replSet info 192.168.59.103:27017 is down (or slow to respond): m31102| 2015-01-26T15:12:56.966+0100 [rsHealthPoll] replSet member 192.168.59.103:27017 is now in state DOWN m30999| 2015-01-26T15:12:57.374+0100 [mongosMain] connection accepted from 127.0.0.1:53208 #14 (10 connections now open) m31100| 2015-01-26T15:12:57.598+0100 [conn259] end connection 192.168.59.3:53135 (6 connections now open) m31100| 2015-01-26T15:12:57.599+0100 [initandlisten] connection accepted from 192.168.59.3:53209 #268 (7 connections now open) m31100| 2015-01-26T15:12:57.789+0100 [conn260] end connection 192.168.59.3:53137 (6 connections now open) m31100| 2015-01-26T15:12:57.790+0100 [initandlisten] connection accepted from 192.168.59.3:53210 #269 (7 connections now open) 2015-01-26T15:12:58.957+0100 [ReplicaSetMonitorWatcher] Socket recv() timeout 192.168.59.103:27017 2015-01-26T15:12:58.957+0100 [ReplicaSetMonitorWatcher] SocketException: remote: 192.168.59.103:27017 error: 9001 socket exception [RECV_TIMEOUT] server [192.168.59.103:27017] 2015-01-26T15:12:58.957+0100 [ReplicaSetMonitorWatcher] DBClientCursor::init call() failed 2015-01-26T15:12:58.957+0100 [ReplicaSetMonitorWatcher] Detected bad connection created at 1422281543936032 microSec, clearing pool for 192.168.59.103:27017 of 0 connections 2015-01-26T15:12:58.957+0100 [ReplicaSetMonitorWatcher] warning: No primary detected for set test-rs0 m30999| 2015-01-26T15:12:59.140+0100 [mongosMain] connection accepted from 127.0.0.1:53215 #15 (11 connections now open) m30999| 2015-01-26T15:13:00.395+0100 [mongosMain] connection accepted from 127.0.0.1:53222 #16 (12 connections now open) m30999| 2015-01-26T15:13:01.061+0100 [conn12] warning: No primary detected for set test-rs0 m31100| 2015-01-26T15:13:01.893+0100 [rsMgr] not electing self, Lowang-MacBook-Pro.local:31102 would veto with 'Lowang-MacBook-Pro.local:31100 has lower priority than 192.168.59.103:27017' m31101| 2015-01-26T15:13:01.957+0100 [rsMgr] not electing self, Lowang-MacBook-Pro.local:31102 would veto with 'Lowang-MacBook-Pro.local:31101 has lower priority than 192.168.59.103:27017' m29000| 2015-01-26T15:13:02.057+0100 [initandlisten] connection accepted from 192.168.59.3:53229 #13 (6 connections now open) m30999| 2015-01-26T15:13:02.159+0100 [mongosMain] connection accepted from 127.0.0.1:53230 #17 (13 connections now open) m30999| 2015-01-26T15:13:03.414+0100 [mongosMain] connection accepted from 127.0.0.1:53234 #18 (14 connections now open) m31102| 2015-01-26T15:13:03.976+0100 [rsHealthPoll] replset info 192.168.59.103:27017 heartbeat failed, retrying m30999| 2015-01-26T15:13:05.173+0100 [mongosMain] connection accepted from 127.0.0.1:53236 #19 (15 connections now open) m30999| 2015-01-26T15:13:06.063+0100 [conn9] warning: No primary detected for set test-rs0 m30999| 2015-01-26T15:13:06.063+0100 [conn9] warning: problem while initially checking shard versions on test-rs0 :: caused by :: 10009 ReplicaSetMonitor no master found for set: test-rs0 m30999| 2015-01-26T15:13:06.063+0100 [conn8] warning: problem while initially checking shard versions on test-rs0 :: caused by :: 10009 ReplicaSetMonitor no master found for set: test-rs0 m30999| 2015-01-26T15:13:06.063+0100 [conn7] warning: problem while initially checking shard versions on test-rs0 :: caused by :: 10009 ReplicaSetMonitor no master found for set: test-rs0 m31100| 2015-01-26T15:13:06.065+0100 [initandlisten] connection accepted from 192.168.59.3:53238 #270 (8 connections now open) m30999| 2015-01-26T15:13:06.065+0100 [conn9] end connection 127.0.0.1:53195 (14 connections now open) m31100| 2015-01-26T15:13:06.065+0100 [conn270] end connection 192.168.59.3:53238 (7 connections now open) m31101| 2015-01-26T15:13:06.065+0100 [initandlisten] connection accepted from 192.168.59.3:53239 #263 (10 connections now open) m31100| 2015-01-26T15:13:06.066+0100 [initandlisten] connection accepted from 192.168.59.3:53240 #271 (8 connections now open) m31101| 2015-01-26T15:13:06.066+0100 [conn263] end connection 192.168.59.3:53239 (9 connections now open) m30999| 2015-01-26T15:13:06.066+0100 [conn8] end connection 127.0.0.1:53194 (13 connections now open) m31100| 2015-01-26T15:13:06.066+0100 [conn271] end connection 192.168.59.3:53240 (7 connections now open) m30999| 2015-01-26T15:13:06.066+0100 [conn7] end connection 127.0.0.1:53188 (13 connections now open) m30999| 2015-01-26T15:13:06.432+0100 [mongosMain] connection accepted from 127.0.0.1:53241 #20 (13 connections now open) m31100| 2015-01-26T15:13:06.896+0100 [rsHealthPoll] replset info 192.168.59.103:27017 heartbeat failed, retrying m31101| 2015-01-26T15:13:06.962+0100 [rsHealthPoll] replset info 192.168.59.103:27017 heartbeat failed, retrying m30999| 2015-01-26T15:13:08.194+0100 [mongosMain] connection accepted from 127.0.0.1:53244 #21 (14 connections now open) m30999| 2015-01-26T15:13:09.453+0100 [mongosMain] connection accepted from 127.0.0.1:53246 #22 (15 connections now open) m30999| 2015-01-26T15:13:11.065+0100 [conn10] warning: No primary detected for set test-rs0 m30999| 2015-01-26T15:13:11.065+0100 [conn10] warning: problem while initially checking shard versions on test-rs0 :: caused by :: 10009 ReplicaSetMonitor no master found for set: test-rs0 m30999| 2015-01-26T15:13:11.065+0100 [conn11] warning: problem while initially checking shard versions on test-rs0 :: caused by :: 10009 ReplicaSetMonitor no master found for set: test-rs0 m30999| 2015-01-26T15:13:11.065+0100 [Balancer] scoped connection to Lowang-MacBook-Pro.local:29000 not being returned to the pool m30999| 2015-01-26T15:13:11.066+0100 [Balancer] caught exception while doing balance: ReplicaSetMonitor no master found for set: test-rs0 m29000| 2015-01-26T15:13:11.066+0100 [conn12] end connection 192.168.59.3:53190 (5 connections now open) m30999| 2015-01-26T15:13:11.066+0100 [conn12] warning: problem while initially checking shard versions on test-rs0 :: caused by :: 10009 ReplicaSetMonitor no master found for set: test-rs0 m31101| 2015-01-26T15:13:11.067+0100 [initandlisten] connection accepted from 192.168.59.3:53250 #264 (10 connections now open) m30999| 2015-01-26T15:13:11.068+0100 [conn10] end connection 127.0.0.1:53198 (14 connections now open) m31101| 2015-01-26T15:13:11.068+0100 [initandlisten] connection accepted from 192.168.59.3:53251 #265 (11 connections now open) m31101| 2015-01-26T15:13:11.068+0100 [conn264] end connection 192.168.59.3:53250 (10 connections now open) m31101| 2015-01-26T15:13:11.068+0100 [initandlisten] connection accepted from 192.168.59.3:53252 #266 (12 connections now open) m31101| 2015-01-26T15:13:11.068+0100 [conn265] end connection 192.168.59.3:53251 (10 connections now open) m31101| 2015-01-26T15:13:11.068+0100 [conn266] end connection 192.168.59.3:53252 (9 connections now open) m30999| 2015-01-26T15:13:11.068+0100 [conn11] end connection 127.0.0.1:53199 (13 connections now open) m30999| 2015-01-26T15:13:11.068+0100 [conn12] end connection 127.0.0.1:53200 (12 connections now open) m30999| 2015-01-26T15:13:11.215+0100 [mongosMain] connection accepted from 127.0.0.1:53253 #23 (13 connections now open) m31100| 2015-01-26T15:13:11.902+0100 [rsMgr] replSet not trying to elect self as responded yea to someone else recently m31100| 2015-01-26T15:13:11.902+0100 [rsMgr] replSet not trying to elect self as responded yea to someone else recently m31100| 2015-01-26T15:13:11.902+0100 [rsMgr] replSet not trying to elect self as responded yea to someone else recently m31100| 2015-01-26T15:13:11.902+0100 [rsMgr] replSet not trying to elect self as responded yea to someone else recently m31101| 2015-01-26T15:13:12.107+0100 [rsMgr] replSet not trying to elect self as responded yea to someone else recently m30999| 2015-01-26T15:13:12.474+0100 [mongosMain] connection accepted from 127.0.0.1:53256 #24 (14 connections now open) m31101| 2015-01-26T15:13:12.899+0100 [rsMgr] replSet not trying to elect self as responded yea to someone else recently m31102| 2015-01-26T15:13:13.012+0100 [rsMgr] replSet I don't see a primary and I can't elect myself m31100| 2015-01-26T15:13:13.012+0100 [rsMgr] replSet not trying to elect self as responded yea to someone else recently m31101| 2015-01-26T15:13:13.758+0100 [rsMgr] replSet not trying to elect self as responded yea to someone else recently 2015-01-26T15:13:13.968+0100 [ReplicaSetMonitorWatcher] warning: No primary detected for set test-rs0 m30999| 2015-01-26T15:13:14.236+0100 [mongosMain] connection accepted from 127.0.0.1:53257 #25 (15 connections now open) m31101| 2015-01-26T15:13:14.709+0100 [rsMgr] replSet info electSelf 1 m31102| 2015-01-26T15:13:14.709+0100 [conn236] replSet info voting yea for Lowang-MacBook-Pro.local:31101 (1) m31100| 2015-01-26T15:13:14.709+0100 [conn268] replSet info voting yea for Lowang-MacBook-Pro.local:31101 (1) m31102| 2015-01-26T15:13:15.014+0100 [conn236] end connection 192.168.59.3:53177 (4 connections now open) m31102| 2015-01-26T15:13:15.014+0100 [conn237] end connection 192.168.59.3:53178 (3 connections now open) m31101| 2015-01-26T15:13:15.014+0100 [conn261] end connection 192.168.59.3:53179 (8 connections now open) m31102| 2015-01-26T15:13:15.015+0100 [initandlisten] connection accepted from 192.168.59.3:53258 #238 (4 connections now open) m31102| 2015-01-26T15:13:15.016+0100 [initandlisten] connection accepted from 192.168.59.3:53260 #239 (5 connections now open) m31101| 2015-01-26T15:13:15.016+0100 [initandlisten] connection accepted from 192.168.59.3:53259 #267 (9 connections now open) m30999| 2015-01-26T15:13:15.494+0100 [mongosMain] connection accepted from 127.0.0.1:53261 #26 (16 connections now open) m31101| 2015-01-26T15:13:15.601+0100 [rsMgr] replSet PRIMARY m31102| 2015-01-26T15:13:15.992+0100 [rsHealthPoll] replset info 192.168.59.103:27017 heartbeat failed, retrying m30999| 2015-01-26T15:13:16.072+0100 [conn15] warning: No primary detected for set test-rs0 m30999| 2015-01-26T15:13:16.072+0100 [conn15] warning: problem while initially checking shard versions on test-rs0 :: caused by :: 10009 ReplicaSetMonitor no master found for set: test-rs0 m30999| 2015-01-26T15:13:16.072+0100 [conn13] warning: problem while initially checking shard versions on test-rs0 :: caused by :: 10009 ReplicaSetMonitor no master found for set: test-rs0 m30999| 2015-01-26T15:13:16.072+0100 [conn14] warning: problem while initially checking shard versions on test-rs0 :: caused by :: 10009 ReplicaSetMonitor no master found for set: test-rs0 m30999| 2015-01-26T15:13:16.072+0100 [conn20] warning: failed to initialize new replica set connection version, will initialize on first use m30999| 2015-01-26T15:13:16.072+0100 [conn2] warning: problem while initially checking shard versions on test-rs0 :: caused by :: 10009 ReplicaSetMonitor no master found for set: test-rs0 m31100| 2015-01-26T15:13:16.075+0100 [initandlisten] connection accepted from 192.168.59.3:53264 #272 (8 connections now open) m31101| 2015-01-26T15:13:16.075+0100 [initandlisten] connection accepted from 192.168.59.3:53266 #268 (10 connections now open) m31100| 2015-01-26T15:13:16.076+0100 [initandlisten] connection accepted from 192.168.59.3:53265 #273 (9 connections now open) m31100| 2015-01-26T15:13:16.077+0100 [conn272] end connection 192.168.59.3:53264 (7 connections now open) m30999| 2015-01-26T15:13:16.077+0100 [conn13] end connection 127.0.0.1:53205 (15 connections now open) m31101| 2015-01-26T15:13:16.077+0100 [initandlisten] connection accepted from 192.168.59.3:53268 #269 (11 connections now open) m31100| 2015-01-26T15:13:16.077+0100 [initandlisten] connection accepted from 192.168.59.3:53267 #274 (9 connections now open) m31101| 2015-01-26T15:13:16.077+0100 [initandlisten] connection accepted from 192.168.59.3:53269 #270 (12 connections now open) m30999| 2015-01-26T15:13:16.077+0100 [conn15] end connection 127.0.0.1:53215 (14 connections now open) m31100| 2015-01-26T15:13:16.077+0100 [conn273] end connection 192.168.59.3:53265 (8 connections now open) m31101| 2015-01-26T15:13:16.077+0100 [initandlisten] connection accepted from 192.168.59.3:53270 #271 (13 connections now open) m30999| 2015-01-26T15:13:16.077+0100 [conn14] end connection 127.0.0.1:53208 (13 connections now open) m31101| 2015-01-26T15:13:16.078+0100 [initandlisten] connection accepted from 192.168.59.3:53271 #272 (14 connections now open) m31101| 2015-01-26T15:13:16.078+0100 [initandlisten] connection accepted from 192.168.59.3:53272 #273 (15 connections now open) m30999| 2015-01-26T15:13:16.084+0100 [conn21] end connection 127.0.0.1:53244 (12 connections now open) m30999| 2015-01-26T15:13:16.084+0100 [conn16] end connection 127.0.0.1:53222 (11 connections now open) m30999| 2015-01-26T15:13:16.085+0100 [conn20] end connection 127.0.0.1:53241 (10 connections now open) m30999| 2015-01-26T15:13:16.085+0100 [conn25] end connection 127.0.0.1:53257 (10 connections now open) m30999| 2015-01-26T15:13:16.085+0100 [conn22] end connection 127.0.0.1:53246 (8 connections now open) m30999| 2015-01-26T15:13:16.085+0100 [conn18] end connection 127.0.0.1:53234 (7 connections now open) m30999| 2015-01-26T15:13:16.085+0100 [conn17] end connection 127.0.0.1:53230 (7 connections now open) m30999| 2015-01-26T15:13:16.085+0100 [conn23] end connection 127.0.0.1:53253 (5 connections now open) m29000| 2015-01-26T15:13:16.078+0100 [initandlisten] connection accepted from 192.168.59.3:53277 #14 (6 connections now open) m31101| 2015-01-26T15:13:16.078+0100 [conn270] remotely refreshing metadata for mongodb_outage_test.data with requested shard version 1|16||54c64644ce5c61cbab9c82e2, current shard version is 0|0||000000000000000000000000, current metadata version is 0|0||000000000000000000000000 m31101| 2015-01-26T15:13:16.078+0100 [conn270] Socket say send() errno:9 Bad file descriptor 192.168.59.3:29000 m31101| 2015-01-26T15:13:16.079+0100 [initandlisten] connection accepted from 192.168.59.3:53273 #274 (16 connections now open) m31101| 2015-01-26T15:13:16.079+0100 [conn273] remotely refreshing metadata for mongodb_outage_test.data with requested shard version 1|16||54c64644ce5c61cbab9c82e2, current shard version is 0|0||000000000000000000000000, current metadata version is 0|0||000000000000000000000000 m31101| 2015-01-26T15:13:16.079+0100 [conn268] remotely refreshing metadata for mongodb_outage_test.data with requested shard version 1|16||54c64644ce5c61cbab9c82e2, current shard version is 0|0||000000000000000000000000, current metadata version is 0|0||000000000000000000000000 m31101| 2015-01-26T15:13:16.080+0100 [conn270] Detected bad connection created at 1422281552342566 microSec, clearing pool for Lowang-MacBook-Pro.local:29000 of 0 connections m31101| 2015-01-26T15:13:16.080+0100 [initandlisten] connection accepted from 192.168.59.3:53274 #275 (17 connections now open) m31101| 2015-01-26T15:13:16.080+0100 [conn270] warning: could not remotely refresh metadata for mongodb_outage_test.data :: caused by :: could not query collection metadata :: caused by :: 9001 socket exception [SEND_ERROR] server [192.168.59.3:29000] m31101| 2015-01-26T15:13:16.080+0100 [conn270] warning: could not refresh metadata for mongodb_outage_test.data with requested shard version 1|16||54c64644ce5c61cbab9c82e2, stored shard version is 0|0||000000000000000000000000 :: caused by :: could not query collection metadata :: caused by :: 9001 socket exception [SEND_ERROR] server [192.168.59.3:29000] m31101| 2015-01-26T15:13:16.080+0100 [conn274] remotely refreshing metadata for mongodb_outage_test.data with requested shard version 1|16||54c64644ce5c61cbab9c82e2, current shard version is 0|0||000000000000000000000000, current metadata version is 0|0||000000000000000000000000 m31101| 2015-01-26T15:13:16.080+0100 [initandlisten] connection accepted from 192.168.59.3:53275 #276 (18 connections now open) m31101| 2015-01-26T15:13:16.080+0100 [initandlisten] connection accepted from 192.168.59.3:53276 #277 (19 connections now open) m31101| 2015-01-26T15:13:16.081+0100 [conn273] collection mongodb_outage_test.data was previously unsharded, new metadata loaded with shard version 1|16||54c64644ce5c61cbab9c82e2 m31101| 2015-01-26T15:13:16.081+0100 [conn273] collection version was loaded at version 1|16||54c64644ce5c61cbab9c82e2, took 2ms m31101| 2015-01-26T15:13:16.082+0100 [conn268] metadata of collection mongodb_outage_test.data already up to date (shard version : 1|16||54c64644ce5c61cbab9c82e2, took 2ms) m31101| 2015-01-26T15:13:16.082+0100 [conn274] metadata of collection mongodb_outage_test.data already up to date (shard version : 1|16||54c64644ce5c61cbab9c82e2, took 1ms) m31100| 2015-01-26T15:13:16.077+0100 [conn274] end connection 192.168.59.3:53267 (7 connections now open) m31100| 2015-01-26T15:13:16.083+0100 [initandlisten] connection accepted from 192.168.59.3:53283 #275 (8 connections now open) m31100| 2015-01-26T15:13:16.084+0100 [initandlisten] connection accepted from 192.168.59.3:53284 #276 (9 connections now open) m31100| 2015-01-26T15:13:16.084+0100 [initandlisten] connection accepted from 192.168.59.3:53285 #277 (10 connections now open) m31100| 2015-01-26T15:13:16.084+0100 [initandlisten] connection accepted from 192.168.59.3:53286 #278 (11 connections now open) m31100| 2015-01-26T15:13:16.084+0100 [initandlisten] connection accepted from 192.168.59.3:53287 #279 (12 connections now open) m31100| 2015-01-26T15:13:16.085+0100 [initandlisten] connection accepted from 192.168.59.3:53288 #280 (13 connections now open) m31100| 2015-01-26T15:13:16.085+0100 [initandlisten] connection accepted from 192.168.59.3:53289 #281 (14 connections now open) m31100| 2015-01-26T15:13:16.085+0100 [initandlisten] connection accepted from 192.168.59.3:53290 #282 (15 connections now open) m29000| 2015-01-26T15:13:16.079+0100 [initandlisten] connection accepted from 192.168.59.3:53278 #15 (7 connections now open) m29000| 2015-01-26T15:13:16.080+0100 [initandlisten] connection accepted from 192.168.59.3:53279 #16 (8 connections now open) m29000| 2015-01-26T15:13:16.080+0100 [initandlisten] connection accepted from 192.168.59.3:53280 #17 (9 connections now open) m29000| 2015-01-26T15:13:16.081+0100 [initandlisten] connection accepted from 192.168.59.3:53281 #18 (10 connections now open) m29000| 2015-01-26T15:13:16.081+0100 [initandlisten] connection accepted from 192.168.59.3:53282 #19 (11 connections now open) m31100| 2015-01-26T15:13:16.106+0100 [initandlisten] connection accepted from 192.168.59.3:53291 #283 (16 connections now open) m30999| 2015-01-26T15:13:16.106+0100 [conn24] end connection 127.0.0.1:53256 (4 connections now open) m31102| 2015-01-26T15:13:17.018+0100 [rsHealthPoll] replSet member Lowang-MacBook-Pro.local:31101 is now in state PRIMARY m31101| 2015-01-26T15:13:17.071+0100 [initandlisten] connection accepted from 192.168.59.3:53292 #278 (20 connections now open) m30999| 2015-01-26T15:13:17.073+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c64b7dce5c61cbab9c835e m30999| 2015-01-26T15:13:17.073+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m31100| 2015-01-26T15:13:17.517+0100 [rsHealthPoll] replSet member Lowang-MacBook-Pro.local:31101 is now in state PRIMARY m31101| 2015-01-26T15:13:21.077+0100 [initandlisten] connection accepted from 192.168.59.3:53295 #279 (21 connections now open) m31100| 2015-01-26T15:13:21.078+0100 [initandlisten] connection accepted from 192.168.59.3:53296 #284 (17 connections now open) m30999| 2015-01-26T15:13:21.078+0100 [conn19] end connection 127.0.0.1:53236 (3 connections now open) m30999| 2015-01-26T15:13:22.911+0100 [conn26] end connection 127.0.0.1:53261 (2 connections now open) m30999| 2015-01-26T15:13:23.081+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c64b83ce5c61cbab9c835f m30999| 2015-01-26T15:13:23.081+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. m30999| 2015-01-26T15:13:23.308+0100 [LockPinger] cluster Lowang-MacBook-Pro.local:29000 pinged successfully at Mon Jan 26 15:13:23 2015 by distributed lock pinger 'Lowang-MacBook-Pro.local:29000/Lowang-MacBook-Pro.local:30999:1422278030:16807', sleeping for 30000ms m31100| 2015-01-26T15:13:23.916+0100 [rsHealthPoll] couldn't connect to 192.168.59.103:27017: couldn't connect to server 192.168.59.103:27017 (192.168.59.103), connection attempt failed m31101| 2015-01-26T15:13:23.977+0100 [rsHealthPoll] replset info 192.168.59.103:27017 heartbeat failed, retrying m31101| 2015-01-26T15:13:25.527+0100 [conn262] end connection 192.168.59.3:53203 (20 connections now open) m31101| 2015-01-26T15:13:25.528+0100 [initandlisten] connection accepted from 192.168.59.3:53304 #280 (21 connections now open) m31100| 2015-01-26T15:13:27.658+0100 [conn268] end connection 192.168.59.3:53209 (16 connections now open) m31100| 2015-01-26T15:13:27.659+0100 [initandlisten] connection accepted from 192.168.59.3:53305 #285 (17 connections now open) m31100| 2015-01-26T15:13:27.847+0100 [conn269] end connection 192.168.59.3:53210 (16 connections now open) m31100| 2015-01-26T15:13:27.848+0100 [initandlisten] connection accepted from 192.168.59.3:53306 #286 (17 connections now open) m31102| 2015-01-26T15:13:28.006+0100 [rsHealthPoll] couldn't connect to 192.168.59.103:27017: couldn't connect to server 192.168.59.103:27017 (192.168.59.103), connection attempt failed m31100| 2015-01-26T15:13:28.917+0100 [rsHealthPoll] couldn't connect to 192.168.59.103:27017: couldn't connect to server 192.168.59.103:27017 (192.168.59.103) failed, connection attempt failed m30999| 2015-01-26T15:13:29.089+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' acquired, ts : 54c64b89ce5c61cbab9c8360 m30999| 2015-01-26T15:13:29.090+0100 [Balancer] distributed lock 'balancer/Lowang-MacBook-Pro.local:30999:1422278030:16807' unlocked. [1] + 97011 suspended (signal) mongo -nodb $