Adaptive mutex spin: 397356 events in 10.108 seconds (39309 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 376116 97% 97% 0.00 974278 256 Bucket (usermode) 10819 2% 100% 0.00 770116 page pv (usermode) 3984 0% 100% 0.00 199657 UPVO entry (usermode) 824 0% 100% 0.00 333225 256 Bucket sdt_probe+0x38 2542 0% 100% 0.00 78721 page pv sdt_probe+0x38 1357 0% 100% 0.00 106372 32 Bucket (usermode) 477 0% 100% 0.00 40181 UPVO entry sdt_probe+0x38 699 0% 100% 0.00 20205 vm reserv domain (usermode) 33 0% 100% 0.00 60656 vm page free queue (usermode) 86 0% 100% 0.00 22539 vm active pagequeue (usermode) 44 0% 100% 0.00 29373 AHCI channel lock (usermode) 5 0% 100% 0.00 139461 sellck (usermode) 15 0% 100% 0.00 37763 kernelpmap sdt_probe+0x38 39 0% 100% 0.00 11733 xhci0 sdt_probe+0x38 8 0% 100% 0.00 54847 32 Bucket sdt_probe+0x38 27 0% 100% 0.00 13324 vm page free queue sdt_probe+0x38 56 0% 100% 0.00 6139 xhci0 (usermode) 22 0% 100% 0.00 15267 ttymtx (usermode) 66 0% 100% 0.00 4825 ttymtx sdt_probe+0x38 13 0% 100% 0.00 19035 kernelpmap (usermode) 22 0% 100% 0.00 9815 so_rcv sdt_probe+0x38 33 0% 100% 0.00 3351 kqueue sdt_probe+0x38 8 0% 100% 0.00 10959 vm active pagequeue sdt_probe+0x38 8 0% 100% 0.00 4761 kqueue (usermode) 2 0% 100% 0.00 17253 bge3 (usermode) 1 0% 100% 0.00 34205 AHCI channel lock sdt_probe+0x38 1 0% 100% 0.00 32960 vm pgcache sdt_probe+0x38 4 0% 100% 0.00 7920 vm reserv domain sdt_probe+0x38 4 0% 100% 0.00 7790 vm pgcache (usermode) 5 0% 100% 0.00 5583 CAM device lock (usermode) 3 0% 100% 0.00 6475 global softdep sdt_probe+0x38 10 0% 100% 0.00 1747 CAM queue lock sdt_probe+0x38 2 0% 100% 0.00 6138 vnode interlock sdt_probe+0x38 1 0% 100% 0.00 10496 FFS sdt_probe+0x38 2 0% 100% 0.00 5112 bufq dirty lock sdt_probe+0x38 2 0% 100% 0.00 4841 CAM doneq (usermode) 1 0% 100% 0.00 8909 so_snd (usermode) 1 0% 100% 0.00 8883 select mtxpool sdt_probe+0x38 1 0% 100% 0.00 8873 struct mount mtx sdt_probe+0x38 2 0% 100% 0.00 3435 global softdep (usermode) 1 0% 100% 0.00 5874 FFS (usermode) 2 0% 100% 0.00 1479 process lock sdt_probe+0x38 1 0% 100% 0.00 2524 bge3 sdt_probe+0x38 1 0% 100% 0.00 2358 CAM queue lock (usermode) 2 0% 100% 0.00 1149 g_vfs sdt_probe+0x38 1 0% 100% 0.00 1297 g_vfs (usermode) 1 0% 100% 0.00 734 CAM device lock sdt_probe+0x38 1 0% 100% 0.00 633 bufq dirty lock (usermode) 1 0% 100% 0.00 535 pipe mutex sdt_probe+0x38 ------------------------------------------------------------------------------- Adaptive mutex block: 2 events in 10.108 seconds (0 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 2 100% 100% 0.00 727670 page pv (usermode) ------------------------------------------------------------------------------- Spin lock spin: 131 events in 10.108 seconds (13 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 11 89% 89% 0.00 983440 smp rendezvous (usermode) 7 7% 96% 0.00 120909 smp rendezvous sdt_probe+0x38 58 1% 98% 0.00 2911 et_hw_mtx sdt_probe+0x38 25 1% 99% 0.00 4350 callout (usermode) 15 0% 99% 0.00 3825 callout sdt_probe+0x38 1 0% 100% 0.00 45077 turnstile chain (usermode) 9 0% 100% 0.00 3292 sleepq chain (usermode) 3 0% 100% 0.00 6186 sleepq chain sdt_probe+0x38 1 0% 100% 0.00 6342 sched lock 65 (usermode) 1 0% 100% 0.00 2594 entropy harvest mutex (usermode) ------------------------------------------------------------------------------- Thread lock spin: 1754 events in 10.108 seconds (174 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 1 35% 35% 0.00 3217847 sched lock 108 (usermode) 1 26% 61% 0.00 2356643 sched lock 18 (usermode) 119 4% 65% 0.00 3036 sched lock 70 (usermode) 101 3% 68% 0.00 2995 sched lock 72 (usermode) 123 3% 72% 0.00 2409 sched lock 78 (usermode) 110 3% 75% 0.00 2530 sched lock 65 (usermode) 109 3% 77% 0.00 2354 sched lock 74 (usermode) 105 3% 80% 0.00 2313 sched lock 66 (usermode) 132 2% 83% 0.00 1664 sched lock 76 (usermode) 107 2% 85% 0.00 1807 sched lock 77 (usermode) 112 2% 87% 0.00 1577 sched lock 69 (usermode) 95 2% 88% 0.00 1789 sched lock 71 (usermode) 103 2% 90% 0.00 1637 sched lock 73 (usermode) 114 2% 92% 0.00 1464 sched lock 75 (usermode) 102 2% 94% 0.00 1540 sched lock 79 (usermode) 99 2% 96% 0.00 1543 sched lock 68 (usermode) 93 2% 97% 0.00 1548 sched lock 67 (usermode) 96 1% 99% 0.00 1379 sched lock 64 (usermode) 1 1% 99% 0.00 55083 sched lock 56 (usermode) 8 0% 99% 0.00 1939 sched lock 106 (usermode) 6 0% 99% 0.00 2147 sched lock 141 (usermode) 5 0% 100% 0.00 2362 sched lock 156 (usermode) 2 0% 100% 0.00 4640 sched lock 135 (usermode) 2 0% 100% 0.00 3585 sched lock 94 (usermode) 1 0% 100% 0.00 6016 sched lock 7 (usermode) 3 0% 100% 0.00 1876 sched lock 107 (usermode) 1 0% 100% 0.00 2573 sched lock 105 (usermode) 1 0% 100% 0.00 2473 sched lock 95 (usermode) 1 0% 100% 0.00 1467 sched lock 122 (usermode) 1 0% 100% 0.00 1447 sched lock 45 (usermode) ------------------------------------------------------------------------------- R/W writer spin on writer: 86 events in 10.108 seconds (9 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 47 51% 51% 0.00 26147 SUrw sdt_probe+0x38 22 44% 95% 0.00 47552 SUrw (usermode) 9 3% 98% 0.00 7020 bufobj interlock (usermode) 7 2% 99% 0.00 5684 bufobj interlock sdt_probe+0x38 1 1% 100% 0.00 17552 tcpinp (usermode) ------------------------------------------------------------------------------- Adaptive mutex hold: 3695994 events in 10.108 seconds (365634 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 552877 57% 57% 0.00 63881 pmap (usermode) 1689173 33% 90% 0.00 12034 page pv (usermode) 970889 6% 96% 0.00 3858 256 Bucket (usermode) 261798 2% 98% 0.00 4030 UPVO entry (usermode) 74675 1% 99% 0.00 6357 vm reserv (usermode) 11096 0% 99% 0.00 11551 vm page free queue (usermode) 33562 0% 99% 0.00 3685 32 Bucket (usermode) 20467 0% 99% 0.00 5775 vm active pagequeue (usermode) 14390 0% 99% 0.00 6684 vnode interlock (usermode) 13208 0% 100% 0.00 4820 vm reserv domain (usermode) 2018 0% 100% 0.00 19949 pipe mutex (usermode) 5464 0% 100% 0.00 6988 process lock (usermode) 9563 0% 100% 0.00 3775 select mtxpool (usermode) 4897 0% 100% 0.00 4815 xhci0 (usermode) 2393 0% 100% 0.00 7300 ttymtx (usermode) 207 0% 100% 0.00 71832 AHCI channel lock (usermode) 2769 0% 100% 0.00 4354 global softdep (usermode) 2838 0% 100% 0.00 3535 kernelpmap (usermode) 2444 0% 100% 0.00 3732 FFS (usermode) 762 0% 100% 0.00 10656 sellck (usermode) 428 0% 100% 0.00 15681 CAM device lock (usermode) 579 0% 100% 0.00 10794 unp (usermode) 792 0% 100% 0.00 6648 socket (usermode) 1270 0% 100% 0.00 3928 so_rcv (usermode) 623 0% 100% 0.00 7593 vm inactive pagequeue (usermode) 239 0% 100% 0.00 19742 umass0 (usermode) 208 0% 100% 0.00 20583 umass1 (usermode) 1210 0% 100% 0.00 3501 so_snd (usermode) 208 0% 100% 0.00 19828 umass2 (usermode) 409 0% 100% 0.00 9862 bufq dirty lock (usermode) 1659 0% 100% 0.00 2421 bounce pages lock (usermode) 796 0% 100% 0.00 4716 vm pgcache (usermode) 165 0% 100% 0.00 18419 ncvn (usermode) 616 0% 100% 0.00 4881 bge3 (usermode) 672 0% 100% 0.00 4245 sigacts (usermode) 993 0% 100% 0.00 2837 kqueue (usermode) 799 0% 100% 0.00 3468 struct mount mtx (usermode) 1885 0% 100% 0.00 1410 devthrd (usermode) 279 0% 100% 0.00 5992 process group (usermode) 123 0% 100% 0.00 12120 ncbuc (usermode) 75 0% 100% 0.00 15743 USB device mutex (usermode) 59 0% 100% 0.00 19128 chacha20_mtx (usermode) 263 0% 100% 0.00 4056 dirhash list (usermode) 615 0% 100% 0.00 1697 XPT topology lock (usermode) 583 0% 100% 0.00 1784 CAM queue lock (usermode) 512 0% 100% 0.00 1644 tcp_hc_entry (usermode) 431 0% 100% 0.00 1732 CAM doneq (usermode) 45 0% 100% 0.00 16091 Giant (usermode) 268 0% 100% 0.00 2570 sleep mtxpool (usermode) 142 0% 100% 0.00 4638 resv obj lock (usermode) 101 0% 100% 0.00 6193 bufq clean subqueue lock (usermode) 96 0% 100% 0.00 6216 RADIX NODE (usermode) 148 0% 100% 0.00 3936 process_exit (usermode) 203 0% 100% 0.00 2724 dev mtx (usermode) 96 0% 100% 0.00 5595 Syncer mtx (usermode) 141 0% 100% 0.00 3540 SLB table (usermode) 20 0% 100% 0.00 23425 umass3 (usermode) 288 0% 100% 0.00 1594 UMA Cross (usermode) 198 0% 100% 0.00 2213 vm pageout lock (usermode) 140 0% 100% 0.00 2998 reseed mutex (usermode) 300 0% 100% 0.00 1372 devstat (usermode) 92 0% 100% 0.00 4382 ncnegl (usermode) 222 0% 100% 0.00 1514 g_vfs (usermode) 140 0% 100% 0.00 2068 bufspace daemon run lock (usermode) 165 0% 100% 0.00 1635 jail mutex (usermode) 76 0% 100% 0.00 3514 struct mount vlist mtx (usermode) 143 0% 100% 0.00 1743 bufq clean lock (usermode) 76 0% 100% 0.00 3002 cred (usermode) 50 0% 100% 0.00 4523 msgbuf lock (usermode) 115 0% 100% 0.00 1607 g_disk_done (usermode) 45 0% 100% 0.00 3979 thread_dtor (usermode) 53 0% 100% 0.00 3317 procid (usermode) 38 0% 100% 0.00 4132 pwddesc (usermode) 81 0% 100% 0.00 1567 DEVCTL (usermode) 31 0% 100% 0.00 3557 itimer lock (usermode) 18 0% 100% 0.00 5064 buffer daemon lock (usermode) 33 0% 100% 0.00 2544 lle_event (usermode) 47 0% 100% 0.00 1677 UMA Slabs 0 (usermode) 17 0% 100% 0.00 3948 malloc-256 (usermode) 12 0% 100% 0.00 5036 umtxql (usermode) 3 0% 100% 0.00 19450 vdbatch (usermode) 20 0% 100% 0.00 2399 igmp_mtx (usermode) 20 0% 100% 0.00 2352 buffer arena-32 (usermode) 9 0% 100% 0.00 4834 MAP ENTRY (usermode) 15 0% 100% 0.00 2840 devfs interlock (usermode) 20 0% 100% 0.00 1981 mld_mtx (usermode) 11 0% 100% 0.00 3575 eventhandler (usermode) 10 0% 100% 0.00 3638 DIRHASH (usermode) 7 0% 100% 0.00 4758 buf free cache (usermode) 16 0% 100% 0.00 1933 process_exec (usermode) 19 0% 100% 0.00 1553 malloc-2048 (usermode) 9 0% 100% 0.00 3237 malloc-128 (usermode) 11 0% 100% 0.00 2643 vnode_list (usermode) 19 0% 100% 0.00 1431 g_bio (usermode) 5 0% 100% 0.00 4951 VM OBJECT (usermode) 7 0% 100% 0.00 3457 thread (usermode) 11 0% 100% 0.00 2183 acqtree (usermode) 1 0% 100% 0.00 23575 unp vp mtxpool (usermode) 11 0% 100% 0.00 1720 spacqtree (usermode) 7 0% 100% 0.00 2354 4 Bucket (usermode) 4 0% 100% 0.00 3900 cdev (usermode) 9 0% 100% 0.00 1619 runningbufspace lock (usermode) 8 0% 100% 0.00 1757 PGRP (usermode) 9 0% 100% 0.00 1545 itimer (usermode) 9 0% 100% 0.00 1528 dirty buf lock (usermode) 4 0% 100% 0.00 3171 clpbuf (usermode) 6 0% 100% 0.00 1761 da_ccb (usermode) 6 0% 100% 0.00 1640 ada_ccb (usermode) 2 0% 100% 0.00 3510 session (usermode) 2 0% 100% 0.00 3024 malloc-64 (usermode) 1 0% 100% 0.00 5900 malloc-384 (usermode) 3 0% 100% 0.00 1795 FFS2 dinode (usermode) 1 0% 100% 0.00 5314 BUF TRIE (usermode) 3 0% 100% 0.00 1620 unp_defer (usermode) 3 0% 100% 0.00 1549 FFS inode (usermode) 1 0% 100% 0.00 4533 dqhlock (usermode) 2 0% 100% 0.00 2015 tcp_log_id_bucket (usermode) 2 0% 100% 0.00 2010 tfo_ccache_entries (usermode) 2 0% 100% 0.00 1996 mlx5_fs_fgs (usermode) 2 0% 100% 0.00 1947 vm map (system) (usermode) 2 0% 100% 0.00 1872 tcp_log_id_node (usermode) 2 0% 100% 0.00 1868 IPsec SA lft_c (usermode) 2 0% 100% 0.00 1865 udp_inpcb (usermode) 2 0% 100% 0.00 1862 AIOLIO (usermode) 1 0% 100% 0.00 3667 128 Bucket (usermode) 2 0% 100% 0.00 1801 swblk (usermode) 2 0% 100% 0.00 1791 dtrace_state_cache (usermode) 2 0% 100% 0.00 1781 rtentry (usermode) 2 0% 100% 0.00 1753 Mountpoints (usermode) 2 0% 100% 0.00 1739 taskq_zone (usermode) 2 0% 100% 0.00 1737 udplite_inpcb (usermode) 2 0% 100% 0.00 1737 AIO (usermode) 2 0% 100% 0.00 1736 tfo (usermode) 2 0% 100% 0.00 1715 tcp_inpcb (usermode) 2 0% 100% 0.00 1693 ksiginfo (usermode) 2 0% 100% 0.00 1688 mlx5_fs_ftes (usermode) 2 0% 100% 0.00 1679 routing nhops (usermode) 2 0% 100% 0.00 1677 sackhole (usermode) 2 0% 100% 0.00 1667 FFS1 dinode (usermode) 2 0% 100% 0.00 1664 so_glabel (usermode) 2 0% 100% 0.00 1649 ipq (usermode) 2 0% 100% 0.00 1647 pipe (usermode) 2 0% 100% 0.00 1634 swpctrie (usermode) 2 0% 100% 0.00 1621 unpcb (usermode) 2 0% 100% 0.00 1611 syncache (usermode) 2 0% 100% 0.00 1577 tcp_log (usermode) 2 0% 100% 0.00 1559 AIOCB (usermode) 2 0% 100% 0.00 1552 ripcb (usermode) 2 0% 100% 0.00 1548 tcpreass (usermode) 2 0% 100% 0.00 1542 hostcache (usermode) 2 0% 100% 0.00 1529 KNOTE (usermode) 1 0% 100% 0.00 1802 mbuf (usermode) 1 0% 100% 0.00 1680 malloc-4096 (usermode) ------------------------------------------------------------------------------- Spin lock hold: 1389212 events in 10.108 seconds (137431 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 1133655 63% 63% 0.00 4410 et_hw_mtx (usermode) 3471 21% 84% 0.00 483105 smp rendezvous (usermode) 11720 1% 85% 0.00 8340 callout (usermode) 10001 1% 86% 0.00 5271 tcsetc (usermode) 5431 1% 86% 0.00 9462 sleepq chain (usermode) 3958 0% 86% 0.00 2796 sched lock 102 (usermode) 1849 0% 86% 0.00 5165 sched lock 56 (usermode) 1875 0% 87% 0.00 5053 sched lock 79 (usermode) 2060 0% 87% 0.00 4483 sched lock 76 (usermode) 2027 0% 87% 0.00 4509 sched lock 70 (usermode) 1984 0% 87% 0.00 4508 sched lock 78 (usermode) 1856 0% 87% 0.00 4784 sched lock 72 (usermode) 1952 0% 87% 0.00 4484 sched lock 75 (usermode) 1924 0% 87% 0.00 4513 sched lock 74 (usermode) 1786 0% 87% 0.00 4828 sched lock 71 (usermode) 1919 0% 87% 0.00 4482 sched lock 73 (usermode) 1883 0% 88% 0.00 4564 sched lock 77 (usermode) 1876 0% 88% 0.00 4526 sched lock 65 (usermode) 1877 0% 88% 0.00 4523 sched lock 69 (usermode) 1841 0% 88% 0.00 4537 sched lock 67 (usermode) 1812 0% 88% 0.00 4579 sched lock 68 (usermode) 1846 0% 88% 0.00 4489 sched lock 66 (usermode) 1792 0% 88% 0.00 4609 sched lock 64 (usermode) 1529 0% 88% 0.00 5390 sched lock 31 (usermode) 2431 0% 88% 0.00 3389 sched lock 107 (usermode) 2337 0% 89% 0.00 3494 sched lock 138 (usermode) 1467 0% 89% 0.00 5533 sched lock 48 (usermode) 1471 0% 89% 0.00 5493 sched lock 55 (usermode) 1360 0% 89% 0.00 5816 sched lock 23 (usermode) 1585 0% 89% 0.00 4930 sched lock 6 (usermode) 1660 0% 89% 0.00 4628 sched lock 42 (usermode) 1315 0% 89% 0.00 5827 sched lock 7 (usermode) 1536 0% 89% 0.00 4981 sched lock 20 (usermode) 1307 0% 89% 0.00 5803 sched lock 16 (usermode) 1369 0% 89% 0.00 5526 sched lock 39 (usermode) 1335 0% 89% 0.00 5634 sched lock 32 (usermode) 1291 0% 90% 0.00 5799 sched lock 24 (usermode) 1342 0% 90% 0.00 5573 sched lock 15 (usermode) 1783 0% 90% 0.00 4180 sched lock 135 (usermode) 1486 0% 90% 0.00 5001 sched lock 30 (usermode) 1524 0% 90% 0.00 4872 sched lock 17 (usermode) 1387 0% 90% 0.00 5349 sched lock 40 (usermode) 1459 0% 90% 0.00 5076 sched lock 60 (usermode) 1529 0% 90% 0.00 4843 sched lock 58 (usermode) 1453 0% 90% 0.00 5044 sched lock 54 (usermode) 1364 0% 90% 0.00 5360 sched lock 47 (usermode) 1423 0% 90% 0.00 5122 sched lock 21 (usermode) 1523 0% 91% 0.00 4785 sched lock 62 (usermode) 1553 0% 91% 0.00 4692 sched lock 12 (usermode) 1533 0% 91% 0.00 4745 sched lock 34 (usermode) 1389 0% 91% 0.00 5169 sched lock 19 (usermode) 1455 0% 91% 0.00 4934 sched lock 3 (usermode) 1936 0% 91% 0.00 3705 sched lock 141 (usermode) 1413 0% 91% 0.00 5070 sched lock 5 (usermode) 1285 0% 91% 0.00 5522 sched lock 8 (usermode) 1427 0% 91% 0.00 4968 sched lock 10 (usermode) 1366 0% 91% 0.00 5187 sched lock 50 (usermode) 1362 0% 91% 0.00 5199 sched lock 18 (usermode) 1750 0% 92% 0.00 4027 sched lock 154 (usermode) 1362 0% 92% 0.00 5169 sched lock 1 (usermode) 1543 0% 92% 0.00 4549 sched lock 95 (usermode) 1414 0% 92% 0.00 4963 sched lock 26 (usermode) 1465 0% 92% 0.00 4773 sched lock 43 (usermode) 1336 0% 92% 0.00 5233 sched lock 0 (usermode) 1343 0% 92% 0.00 5193 sched lock 51 (usermode) 1345 0% 92% 0.00 5167 sched lock 2 (usermode) 1313 0% 92% 0.00 5267 sched lock 49 (usermode) 1785 0% 92% 0.00 3866 sched lock 108 (usermode) 1400 0% 92% 0.00 4920 sched lock 28 (usermode) 1364 0% 93% 0.00 5039 sched lock 29 (usermode) 1303 0% 93% 0.00 5273 sched lock 22 (usermode) 1663 0% 93% 0.00 4127 sched lock 156 (usermode) 1338 0% 93% 0.00 5119 sched lock 63 (usermode) 1293 0% 93% 0.00 5292 sched lock 52 (usermode) 1284 0% 93% 0.00 5263 sched lock 53 (usermode) 1611 0% 93% 0.00 4192 sched lock 105 (usermode) 1291 0% 93% 0.00 5228 sched lock 4 (usermode) 1333 0% 93% 0.00 5045 sched lock 35 (usermode) 1354 0% 93% 0.00 4957 sched lock 120 (usermode) 1313 0% 93% 0.00 5100 sched lock 33 (usermode) 1641 0% 93% 0.00 4067 sched lock 147 (usermode) 1318 0% 94% 0.00 5059 sched lock 38 (usermode) 1296 0% 94% 0.00 5136 sched lock 61 (usermode) 1284 0% 94% 0.00 5170 sched lock 27 (usermode) 1360 0% 94% 0.00 4877 sched lock 11 (usermode) 1284 0% 94% 0.00 5164 sched lock 25 (usermode) 1359 0% 94% 0.00 4873 sched lock 87 (usermode) 1309 0% 94% 0.00 5046 sched lock 37 (usermode) 1295 0% 94% 0.00 5083 sched lock 57 (usermode) 1138 0% 94% 0.00 5762 sched lock 80 (usermode) 1284 0% 94% 0.00 5095 sched lock 59 (usermode) 1328 0% 94% 0.00 4917 sched lock 44 (usermode) 1310 0% 94% 0.00 4971 sched lock 9 (usermode) 1284 0% 95% 0.00 5070 sched lock 36 (usermode) 1348 0% 95% 0.00 4827 sched lock 88 (usermode) 1300 0% 95% 0.00 4993 sched lock 13 (usermode) 1442 0% 95% 0.00 4479 sched lock 125 (usermode) 1343 0% 95% 0.00 4809 sched lock 46 (usermode) 1203 0% 95% 0.00 5356 sched lock 127 (usermode) 1481 0% 95% 0.00 4343 sched lock 133 (usermode) 1391 0% 95% 0.00 4609 sched lock 106 (usermode) 1292 0% 95% 0.00 4946 sched lock 152 (usermode) 1424 0% 95% 0.00 4485 sched lock 146 (usermode) 1170 0% 95% 0.00 5452 sched lock 103 (usermode) 1303 0% 95% 0.00 4892 sched lock 45 (usermode) 1294 0% 96% 0.00 4922 sched lock 41 (usermode) 1205 0% 96% 0.00 5268 sched lock 96 (usermode) 1284 0% 96% 0.00 4943 sched lock 14 (usermode) 1163 0% 96% 0.00 5429 sched lock 111 (usermode) 1136 0% 96% 0.00 5553 sched lock 151 (usermode) 1136 0% 96% 0.00 5542 sched lock 119 (usermode) 1328 0% 96% 0.00 4740 sched lock 122 (usermode) 1201 0% 96% 0.00 5239 sched lock 128 (usermode) 1151 0% 96% 0.00 5461 sched lock 112 (usermode) 1159 0% 96% 0.00 5413 sched lock 136 (usermode) 1136 0% 96% 0.00 5516 sched lock 143 (usermode) 1149 0% 96% 0.00 5448 sched lock 104 (usermode) 1145 0% 96% 0.00 5442 sched lock 144 (usermode) 1308 0% 97% 0.00 4763 sched lock 124 (usermode) 1324 0% 97% 0.00 4661 sched lock 90 (usermode) 1309 0% 97% 0.00 4708 sched lock 137 (usermode) 1318 0% 97% 0.00 4658 sched lock 149 (usermode) 1229 0% 97% 0.00 4954 sched lock 94 (usermode) 1284 0% 97% 0.00 4729 sched lock 123 (usermode) 1206 0% 97% 0.00 5032 sched lock 150 (usermode) 1236 0% 97% 0.00 4894 sched lock 132 (usermode) 1186 0% 97% 0.00 5086 sched lock 114 (usermode) 1220 0% 97% 0.00 4929 sched lock 159 (usermode) 1134 0% 97% 0.00 5286 sched lock 126 (usermode) 1206 0% 97% 0.00 4969 sched lock 101 (usermode) 1163 0% 97% 0.00 5149 sched lock 110 (usermode) 1137 0% 98% 0.00 5247 sched lock 134 (usermode) 1136 0% 98% 0.00 5240 sched lock 118 (usermode) 1136 0% 98% 0.00 5233 sched lock 142 (usermode) 1136 0% 98% 0.00 5230 sched lock 100 (usermode) 1165 0% 98% 0.00 5085 sched lock 117 (usermode) 1181 0% 98% 0.00 5015 sched lock 145 (usermode) 1214 0% 98% 0.00 4876 sched lock 82 (usermode) 1172 0% 98% 0.00 5050 sched lock 153 (usermode) 1135 0% 98% 0.00 5212 sched lock 121 (usermode) 1136 0% 98% 0.00 5206 sched lock 97 (usermode) 1149 0% 98% 0.00 5147 sched lock 116 (usermode) 1137 0% 98% 0.00 5188 sched lock 86 (usermode) 1136 0% 98% 0.00 5192 sched lock 158 (usermode) 1188 0% 98% 0.00 4960 sched lock 115 (usermode) 1149 0% 99% 0.00 5126 sched lock 140 (usermode) 1165 0% 99% 0.00 5043 sched lock 129 (usermode) 1136 0% 99% 0.00 5170 sched lock 148 (usermode) 1136 0% 99% 0.00 5169 sched lock 98 (usermode) 1137 0% 99% 0.00 5154 sched lock 130 (usermode) 1136 0% 99% 0.00 5153 sched lock 139 (usermode) 1138 0% 99% 0.00 5139 sched lock 85 (usermode) 1137 0% 99% 0.00 5128 sched lock 92 (usermode) 1138 0% 99% 0.00 5121 sched lock 81 (usermode) 1135 0% 99% 0.00 5125 sched lock 113 (usermode) 1144 0% 99% 0.00 5083 sched lock 109 (usermode) 1137 0% 99% 0.00 5108 sched lock 93 (usermode) 1137 0% 99% 0.00 5107 sched lock 157 (usermode) 1136 0% 100% 0.00 5107 sched lock 131 (usermode) 1136 0% 100% 0.00 5103 sched lock 99 (usermode) 1137 0% 100% 0.00 5097 sched lock 89 (usermode) 1136 0% 100% 0.00 5099 sched lock 155 (usermode) 1139 0% 100% 0.00 5072 sched lock 84 (usermode) 1138 0% 100% 0.00 5068 sched lock 91 (usermode) 1137 0% 100% 0.00 5061 sched lock 83 (usermode) 567 0% 100% 0.00 3708 bge_taskq (usermode) 61 0% 100% 0.00 18625 process slock (usermode) 196 0% 100% 0.00 2955 entropy harvest mutex (usermode) 42 0% 100% 0.00 6742 pstatl (usermode) 4 0% 100% 0.00 45931 turnstile lock (usermode) 4 0% 100% 0.00 37053 turnstile chain (usermode) 11 0% 100% 0.00 5055 ntp (usermode) 5 0% 100% 0.00 4139 td_contested (usermode) ------------------------------------------------------------------------------- R/W writer hold: 269568 events in 10.108 seconds (26668 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 259836 98% 98% 0.00 12199 vmobject (usermode) 6385 1% 99% 0.00 5395 SUrw (usermode) 3006 0% 100% 0.00 4280 bufobj interlock (usermode) 214 0% 100% 0.00 23678 tcpinp (usermode) 11 0% 100% 0.00 23142 lle (usermode) 56 0% 100% 0.00 3741 vfs hash (usermode) 34 0% 100% 0.00 4580 tidhash (usermode) 22 0% 100% 0.00 2215 nd6 list (usermode) 3 0% 100% 0.00 1853 unp_link_rwlock (usermode) 1 0% 100% 0.00 1949 uidinfo hash (usermode) ------------------------------------------------------------------------------- R/W reader hold: 2918032 events in 10.108 seconds (288672 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 2891209 64% 64% 0.00 7811 pte eviction (usermode) 26466 36% 100% 0.00 472208 vmobject (usermode) 181 0% 100% 0.00 4138 vfs hash (usermode) 153 0% 100% 0.00 3567 vnet_rwlock (usermode) 11 0% 100% 0.00 6598 tidhash (usermode) 8 0% 100% 0.00 7919 bufobj interlock (usermode) 2 0% 100% 0.00 30951 udpinp (usermode) 2 0% 100% 0.00 2187 helper list lock (usermode) ------------------------------------------------------------------------------- SX shared hold: 124908 events in 10.108 seconds (12357 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 122947 100% 100% 0.00 98576 vm map (user) (usermode) 4 0% 100% 0.00 8470593 killpg racer (usermode) 259 0% 100% 0.00 37141 dirhash (usermode) 1530 0% 100% 0.00 1949 filedesc structure (usermode) 167 0% 100% 0.00 1913 sysctl string handler (usermode) 1 0% 100% 0.00 3977 pidhash (usermode) ------------------------------------------------------------------------------- SX exclusive hold: 9136 events in 10.108 seconds (904 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 3625 99% 99% 0.00 3666500 vm map (user) (usermode) 1499 0% 99% 0.00 27028 dtrace_lock (usermode) 82 0% 99% 0.00 350765 CMDQ lock (usermode) 27 0% 100% 0.00 715606 USB config SX lock (usermode) 2915 0% 100% 0.00 6164 filedesc structure (usermode) 66 0% 100% 0.00 149903 USB suspend and resume SX lock (usermode) 39 0% 100% 0.00 244326 USB control transfer SX lock (usermode) 103 0% 100% 0.00 91832 proctree (usermode) 270 0% 100% 0.00 25931 so_snd_sx (usermode) 201 0% 100% 0.00 9031 so_rcv_sx (usermode) 172 0% 100% 0.00 7004 dirhash (usermode) 42 0% 100% 0.00 5328 fasttrap_provs.fth_table[i].ftb_mtx (usermode) 34 0% 100% 0.00 5688 allproc (usermode) 35 0% 100% 0.00 4965 pidhash (usermode) 10 0% 100% 0.00 14560 umareclaim (usermode) 15 0% 100% 0.00 3753 devfsmount (usermode) 1 0% 100% 0.00 22688 killpg racer (usermode) ------------------------------------------------------------------------------- lockmgr shared hold: 9786 events in 10.108 seconds (968 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 9772 100% 100% 0.00 28118 ufs (usermode) 14 0% 100% 0.00 10972 devfs (usermode) ------------------------------------------------------------------------------- lockmgr exclusive hold: 2965 events in 10.108 seconds (293 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 2215 49% 49% 0.00 114368 bufwait (usermode) 702 47% 95% 0.00 346507 ufs (usermode) 33 4% 100% 0.00 662959 rename (usermode) 3 0% 100% 0.00 505179 pbufwait (usermode) 12 0% 100% 0.00 9355 devfs (usermode) -------------------------------------------------------------------------------