| Message ID | 6116275.lOV4Wx5bFT@rjwysocki.net |
|---|---|
| Headers | show |
| Series | cpuidle: teo: Cleanups and very frequent wakeups handling update | expand |
On 1/13/25 18:32, Rafael J. Wysocki wrote: > Hi Everyone, > > This supersedes > > https://lore.kernel.org/linux-pm/4953183.GXAFRqVoOG@rjwysocki.net/ > > but because the majority of patches in it are new, I've decided to count > version numbers back from 1. > > This addresses a relatively recently added inconsistency in behavior of the teo > governor regarding the handling of very frequent wakeups handling (patch [7/9]) > and makes some other changes that may be regarded as cleanups. > > Please review. Hi Rafael, that looks promising. I'll review the individual patches in detail now, but I let a few tests run overnight and can report that there's no significant behaviour change with the series on an arm64 (no polling state, rk3399), which is my expected result. I'll get something running on a system with a polling state as well. (I don't have a POWER system, so that will just be x86, adding Aboorva.) Tested-by: Christian Loehle <christian.loehle@arm.com> for the entire series I'll just dump some of the raw results here for reference. teo-X is teo with everything up to X/9 applied. teo-m is mainline, teo is the entire series (equivalent to teo-9). Everything is done with fio direct random4k read. Range of real devices with varying latency, mapper/slow is 51ms timer delay, nullb0 is essentially CPU-bound, few intercepts expected. -- device gov iter iops idles idle_misses idle_miss_ratio belows aboves mmcblk1 menu 0 1284 344126 106309 0.309 89264 17045 mmcblk1 menu 1 1340 358646 109469 0.305 91726 17743 mmcblk1 menu 2 1316 348812 106773 0.306 89845 16928 mmcblk1 teo 0 2023 559551 30053 0.054 2962 27091 mmcblk1 teo 1 2007 548422 25816 0.047 2113 23703 mmcblk1 teo 2 2010 557120 29434 0.053 2782 26652 mmcblk1 teo-m 0 2010 542790 23336 0.043 1720 21616 mmcblk1 teo-m 1 2009 554912 29239 0.053 2750 26489 mmcblk1 teo-m 2 2013 548384 26444 0.048 2180 24264 mmcblk1 teo-1 0 2030 524056 3571 0.007 610 2961 mmcblk1 teo-1 1 2022 559304 27074 0.048 3373 23701 mmcblk1 teo-1 2 2009 554744 27406 0.049 2964 24442 mmcblk1 teo-2 0 2012 542886 18233 0.034 2140 16093 mmcblk1 teo-2 1 2010 554094 27680 0.050 2957 24723 mmcblk1 teo-2 2 2010 555222 28549 0.051 3085 25464 mmcblk1 teo-3 0 2011 556816 28708 0.052 3076 25632 mmcblk1 teo-3 1 2015 546998 20280 0.037 2414 17866 mmcblk1 teo-3 2 2020 549758 20802 0.038 2662 18140 mmcblk1 teo-4 0 2026 536464 11692 0.022 1457 10235 mmcblk1 teo-4 1 2031 557978 25464 0.046 2928 22536 mmcblk1 teo-4 2 2012 553812 26323 0.048 2943 23380 mmcblk1 teo-5 0 2021 557966 3454 0.006 3003 451 mmcblk1 teo-5 1 2027 546630 18400 0.034 2303 16097 mmcblk1 teo-5 2 2012 554390 28013 0.051 2990 25023 mmcblk1 teo-6 0 2037 539956 12764 0.024 1701 11063 mmcblk1 teo-6 1 2019 556202 29074 0.052 3071 26003 mmcblk1 teo-6 2 2017 555004 26814 0.048 2869 23945 mmcblk1 teo-7 0 2028 545802 23806 0.044 1625 22181 mmcblk1 teo-7 1 2019 557030 29913 0.054 2990 26923 mmcblk1 teo-7 2 2018 553236 28534 0.052 2657 25877 mmcblk1 teo-8 0 2018 554662 28595 0.052 2660 25935 mmcblk1 teo-8 1 2014 554646 28947 0.052 2721 26226 mmcblk1 teo-8 2 2014 543828 29379 0.054 2915 26464 mmcblk1 teo-9 0 2013 555356 28867 0.052 2719 26148 mmcblk1 teo-9 1 2015 557254 29709 0.053 2836 26873 mmcblk1 teo-9 2 2030 559272 29321 0.052 3052 26269 mmcblk2 menu 0 2885 434990 140864 0.324 123507 17357 mmcblk2 menu 1 2766 450918 152931 0.339 135384 17547 mmcblk2 menu 2 2960 407712 125573 0.308 109224 16349 mmcblk2 teo 0 5677 741006 24131 0.033 2233 21898 mmcblk2 teo 1 5652 714494 24412 0.034 2680 21732 mmcblk2 teo 2 5660 752410 20497 0.027 1809 18688 mmcblk2 teo-m 0 5683 855232 23454 0.027 1934 21520 mmcblk2 teo-m 1 5588 517234 32016 0.062 5443 26573 mmcblk2 teo-m 2 5654 668648 29108 0.044 3171 25937 mmcblk2 teo-1 0 5666 773452 25740 0.033 3430 22310 mmcblk2 teo-1 1 5638 668190 27619 0.041 4150 23469 mmcblk2 teo-1 2 5689 866340 28191 0.033 3122 25069 mmcblk2 teo-2 0 5769 859462 2116 0.002 1835 281 mmcblk2 teo-2 1 5577 611956 25490 0.042 8647 16843 mmcblk2 teo-2 2 5665 721308 25826 0.036 2876 22950 mmcblk2 teo-3 0 5677 805946 30345 0.038 3877 26468 mmcblk2 teo-3 1 5752 855840 14392 0.017 2588 11804 mmcblk2 teo-3 2 5740 825580 27403 0.033 3750 23653 mmcblk2 teo-4 0 5664 777848 3435 0.004 3229 206 mmcblk2 teo-4 1 5660 728926 25796 0.035 2871 22925 mmcblk2 teo-4 2 5689 774342 7844 0.010 2838 5006 mmcblk2 teo-5 0 5672 781618 2733 0.003 2551 182 mmcblk2 teo-5 1 5693 866530 28658 0.033 3332 25326 mmcblk2 teo-5 2 5687 837294 12172 0.015 2338 9834 mmcblk2 teo-6 0 5859 892798 5226 0.006 3566 1660 mmcblk2 teo-6 1 5696 864197 24962 0.029 2796 22166 mmcblk2 teo-6 2 5671 788200 26612 0.034 3157 23455 mmcblk2 teo-7 0 5617 525864 30143 0.057 3739 26404 mmcblk2 teo-7 1 5732 802794 25553 0.032 3240 22313 mmcblk2 teo-7 2 5838 858620 12661 0.015 188 12473 mmcblk2 teo-8 0 5696 780894 24440 0.031 2928 21512 mmcblk2 teo-8 1 5868 862794 12978 0.015 187 12791 mmcblk2 teo-8 2 5607 660124 31632 0.048 6728 24904 mmcblk2 teo-9 0 5652 669318 28330 0.042 2692 25638 mmcblk2 teo-9 1 5657 710892 29291 0.041 3233 26058 mmcblk2 teo-9 2 5594 740988 37086 0.050 10275 26811 nvme0n1 menu 0 4775 384186 70675 0.184 54346 16329 nvme0n1 menu 1 5211 409374 69545 0.170 53484 16061 nvme0n1 menu 2 7213 535088 81312 0.152 64253 17059 nvme0n1 teo 0 10655 754882 29476 0.039 2960 26516 nvme0n1 teo 1 10627 755322 29324 0.039 3011 26313 nvme0n1 teo 2 10559 749498 29850 0.040 3265 26585 nvme0n1 teo-m 0 10353 738802 29937 0.041 2996 26941 nvme0n1 teo-m 1 10629 748148 27076 0.036 2349 24727 nvme0n1 teo-m 2 10477 743256 28193 0.038 2645 25548 nvme0n1 teo-1 0 10947 772905 24896 0.032 3150 21746 nvme0n1 teo-1 1 10412 732894 21868 0.030 2423 19445 nvme0n1 teo-1 2 10530 748377 26338 0.035 3479 22859 nvme0n1 teo-2 0 10570 751961 5195 0.007 2853 2342 nvme0n1 teo-2 1 10482 732428 18667 0.025 2102 16565 nvme0n1 teo-2 2 10829 768292 26400 0.034 3189 23211 nvme0n1 teo-3 0 10493 746638 26371 0.035 3320 23051 nvme0n1 teo-3 1 10445 742924 27871 0.038 3027 24844 nvme0n1 teo-3 2 10920 775112 25646 0.033 3520 22126 nvme0n1 teo-4 0 10500 734792 18125 0.025 2283 15842 nvme0n1 teo-4 1 11091 783828 25904 0.033 3031 22873 nvme0n1 teo-4 2 10413 736248 23786 0.032 2660 21126 nvme0n1 teo-5 0 10479 744340 26933 0.036 3096 23837 nvme0n1 teo-5 1 10740 764188 27065 0.035 3062 24003 nvme0n1 teo-5 2 10500 747060 25962 0.035 3233 22729 nvme0n1 teo-6 0 10715 757772 2186 0.003 1872 314 nvme0n1 teo-6 1 10485 734878 17620 0.024 2174 15446 nvme0n1 teo-6 2 10478 744106 26909 0.036 3007 23902 nvme0n1 teo-7 0 10549 750022 30416 0.041 3320 27096 nvme0n1 teo-7 1 10611 752182 29877 0.040 3332 26545 nvme0n1 teo-7 2 11170 791572 30790 0.039 3453 27337 nvme0n1 teo-8 0 10622 752523 29002 0.039 2745 26257 nvme0n1 teo-8 1 10424 739904 28641 0.039 2756 25885 nvme0n1 teo-8 2 10533 731440 22101 0.030 1280 20821 nvme0n1 teo-9 0 10367 727768 24895 0.034 1820 23075 nvme0n1 teo-9 1 10815 766230 29378 0.038 3136 26242 nvme0n1 teo-9 2 10426 739224 28173 0.038 2563 25610 sda menu 0 872 526240 175645 0.334 159373 16272 sda menu 1 900 536434 188122 0.351 170749 17373 sda menu 2 901 534826 189778 0.355 171745 18033 sda teo 0 1687 999108 16015 0.016 1249 14766 sda teo 1 1668 1007551 31316 0.031 5809 25507 sda teo 2 1682 985830 18784 0.019 1003 17781 sda teo-m 0 1676 969984 25825 0.027 3191 22634 sda teo-m 1 1682 995266 30340 0.030 4040 26300 sda teo-m 2 1681 940562 20830 0.022 1508 19322 sda teo-1 0 1680 996214 15553 0.016 2192 13361 sda teo-1 1 1674 1050168 28033 0.027 4011 24022 sda teo-1 2 1665 808202 35292 0.044 8873 26419 sda teo-2 0 1689 1013360 546 0.001 243 303 sda teo-2 1 1672 1033806 29432 0.028 4363 25069 sda teo-2 2 1667 1046100 31110 0.030 6395 24715 sda teo-3 0 1625 922694 40891 0.044 22925 17966 sda teo-3 1 1670 894480 27946 0.031 3045 24901 sda teo-3 2 1658 1009366 28514 0.028 8887 19627 sda teo-4 0 1674 977280 3605 0.004 3279 326 sda teo-4 1 1677 960990 2861 0.003 1058 1803 sda teo-4 2 1660 1016592 33894 0.033 7687 26207 sda teo-5 0 1678 1033470 16996 0.016 2163 14833 sda teo-5 1 1667 873077 25011 0.029 4393 20618 sda teo-5 2 1672 1042020 24589 0.024 3858 20731 sda teo-6 0 1680 962686 7255 0.008 1797 5458 sda teo-6 1 1682 1055472 28602 0.027 5081 23521 sda teo-6 2 1675 989244 19110 0.019 5620 13490 sda teo-7 0 1618 1001000 51178 0.051 26011 25167 sda teo-7 1 1666 1047310 34032 0.032 6520 27512 sda teo-7 2 1676 982046 28788 0.029 2836 25952 sda teo-8 0 1618 975930 52929 0.054 26386 26543 sda teo-8 1 1678 1023796 16182 0.016 2191 13991 sda teo-8 2 1633 981693 43520 0.044 18782 24738 sda teo-9 0 1661 1029494 32423 0.031 5855 26568 sda teo-9 1 1678 969400 17843 0.018 2763 15080 sda teo-9 2 1679 1055060 32288 0.031 5607 26681 mtdblock3 menu 0 180 278442 80547 0.289 65714 14833 mtdblock3 menu 1 167 379840 109168 0.287 93460 15708 mtdblock3 menu 2 155 373948 126262 0.338 107799 18463 mtdblock3 teo 0 256 663190 29333 0.044 4836 24497 mtdblock3 teo 1 257 808114 29394 0.036 2742 26652 mtdblock3 teo 2 257 456150 25812 0.057 2468 23344 mtdblock3 teo-m 0 257 624492 24678 0.040 2326 22352 mtdblock3 teo-m 1 256 734548 31165 0.042 4307 26858 mtdblock3 teo-m 2 257 812004 30479 0.038 3510 26969 mtdblock3 teo-1 0 253 759708 34146 0.045 10081 24065 mtdblock3 teo-1 1 254 730160 23105 0.032 7156 15949 mtdblock3 teo-1 2 253 721558 32848 0.046 10028 22820 mtdblock3 teo-2 0 257 666426 4948 0.007 782 4166 mtdblock3 teo-2 1 256 682046 25126 0.037 4886 20240 mtdblock3 teo-2 2 253 639950 36029 0.056 11074 24955 mtdblock3 teo-3 0 252 709122 37198 0.052 12603 24595 mtdblock3 teo-3 1 257 709680 30000 0.042 3670 26330 mtdblock3 teo-3 2 254 540408 32696 0.061 9035 23661 mtdblock3 teo-4 0 256 442356 28281 0.064 3785 24496 mtdblock3 teo-4 1 254 588362 16934 0.029 5663 11271 mtdblock3 teo-4 2 257 628776 28300 0.045 3667 24633 mtdblock3 teo-5 0 250 762594 43008 0.056 16752 26256 mtdblock3 teo-5 1 256 586098 29744 0.051 4035 25709 mtdblock3 teo-5 2 262 903736 375 0.000 139 236 mtdblock3 teo-6 0 250 795274 42742 0.054 17160 25582 mtdblock3 teo-6 1 256 800172 28195 0.035 4016 24179 mtdblock3 teo-6 2 257 523365 26260 0.050 2788 23472 mtdblock3 teo-7 0 261 944626 31616 0.033 5090 26526 mtdblock3 teo-7 1 258 647048 28997 0.045 2728 26269 mtdblock3 teo-7 2 260 901858 18521 0.021 3698 14823 mtdblock3 teo-8 0 256 914076 27272 0.030 6844 20428 mtdblock3 teo-8 1 256 809113 31696 0.039 4672 27024 mtdblock3 teo-8 2 252 798396 40223 0.050 13270 26953 mtdblock3 teo-9 0 259 726886 30515 0.042 4002 26513 mtdblock3 teo-9 1 262 892854 9411 0.011 1686 7725 mtdblock3 teo-9 2 259 909528 16583 0.018 3054 13529 nullb0 menu 0 107364 86086 20668 0.240 2604 18064 nullb0 menu 1 107701 85072 20251 0.238 2808 17443 nullb0 menu 2 107976 85178 20591 0.242 3063 17528 nullb0 teo 0 105758 84464 28872 0.342 3429 25443 nullb0 teo 1 106892 87122 30255 0.347 3722 26533 nullb0 teo 2 107080 83374 27911 0.335 2918 24993 nullb0 teo-m 0 107702 88337 29824 0.338 2956 26868 nullb0 teo-m 1 108218 88130 29710 0.337 3038 26672 nullb0 teo-m 2 106993 86866 30033 0.346 3664 26369 nullb0 teo-1 0 106416 82936 23635 0.285 3186 20449 nullb0 teo-1 1 107570 87425 27682 0.317 3220 24462 nullb0 teo-1 2 107832 75656 19434 0.257 2775 16659 nullb0 teo-2 0 106320 88182 28603 0.324 3213 25390 nullb0 teo-2 1 107316 79968 21643 0.271 3081 18562 nullb0 teo-2 2 108007 84018 25244 0.300 3034 22210 nullb0 teo-3 0 106989 85160 24558 0.288 3601 20957 nullb0 teo-3 1 107325 83582 24994 0.299 2963 22031 nullb0 teo-3 2 107063 77066 18978 0.246 2913 16065 nullb0 teo-4 0 107990 86156 25012 0.290 3404 21608 nullb0 teo-4 1 107352 75750 15096 0.199 2671 12425 nullb0 teo-4 2 107685 87644 26047 0.297 3825 22222 nullb0 teo-5 0 106687 87788 13428 0.153 2937 10491 nullb0 teo-5 1 107820 86442 25703 0.297 3686 22017 nullb0 teo-5 2 108319 81498 22895 0.281 2713 20182 nullb0 teo-6 0 107784 84364 24239 0.287 3459 20780 nullb0 teo-6 1 108762 83854 24017 0.286 3289 20728 nullb0 teo-6 2 106024 88238 27778 0.315 3253 24525 nullb0 teo-7 0 106041 86354 30076 0.348 3420 26656 nullb0 teo-7 1 108039 79564 26805 0.337 2924 23881 nullb0 teo-7 2 107389 88264 30713 0.348 3156 27557 nullb0 teo-8 0 106465 87924 29679 0.338 2958 26721 nullb0 teo-8 1 107215 81348 27038 0.332 2916 24122 nullb0 teo-8 2 108238 87070 29284 0.336 2971 26313 nullb0 teo-9 0 107220 84508 28894 0.342 3411 25483 nullb0 teo-9 1 107400 86920 29945 0.345 3579 26366 nullb0 teo-9 2 107845 87686 30439 0.347 3784 26655 mapper/dm-slow menu 0 19 85216 20060 0.235 2478 17582 mapper/dm-slow teo 0 19 86454 28959 0.335 2860 26099 mapper/dm-slow teo-m 0 19 84740 28056 0.331 2582 25474 mapper/dm-slow teo-1 0 19 88698 28945 0.326 3210 25735 mapper/dm-slow teo-2 0 19 83138 24326 0.293 2589 21737 mapper/dm-slow teo-3 0 19 86416 27274 0.316 2998 24276 mapper/dm-slow teo-4 0 19 89286 29369 0.329 3151 26218 mapper/dm-slow teo-5 0 19 88516 29213 0.330 3163 26050 mapper/dm-slow teo-6 0 19 87896 24675 0.281 3113 21562 mapper/dm-slow teo-7 0 19 83986 27976 0.333 2788 25188 mapper/dm-slow teo-8 0 19 88072 29667 0.337 2962 26705 mapper/dm-slow teo-9 0 19 87260 29428 0.337 2923 26505
On Wed, 2025-01-15 at 14:52 +0000, Christian Loehle wrote: > On 1/13/25 18:32, Rafael J. Wysocki wrote: > > Hi Everyone, > > > > This supersedes > > > > https://lore.kernel.org/linux-pm/4953183.GXAFRqVoOG@rjwysocki.net/ > > > > but because the majority of patches in it are new, I've decided to count > > version numbers back from 1. > > > > This addresses a relatively recently added inconsistency in behavior of the teo > > governor regarding the handling of very frequent wakeups handling (patch [7/9]) > > and makes some other changes that may be regarded as cleanups. > > > > Please review. > > Hi Rafael, > that looks promising. I'll review the individual patches in detail now, but > I let a few tests run overnight and can report that there's no significant > behaviour change with the series on an arm64 (no polling state, rk3399), which > is my expected result. > > I'll get something running on a system with a polling state as well. > (I don't have a POWER system, so that will just be x86, adding Aboorva.) Christian, Thanks for adding me to the thread. Rafael, I did some tests with the patch on a Pseries Power10 system: Here are the system details: -------------------------------------------------------------------------------- Architecture: ppc64le Byte Order: Little Endian CPU(s): 48 On-line CPU(s) list: 0-47 Model name: POWER10 (architected), altivec supported Model: 2.0 (pvr 0080 0200) Thread(s) per core: 8 Core(s) per socket: 6 Socket(s): 1 Physical sockets: 8 Physical chips: 1 Physical cores/chip: 10 -------------------------------------------------------------------------------- # cpupower idle-info CPUidle driver: pseries_idle CPUidle governor: menu analyzing CPU 5: Number of idle states: 2 Available idle states: snooze Shared Cede snooze: Flags/Description: snooze Latency: 0 Usage: 1411724 Duration: 27481504 Shared Cede: Flags/Description: Shared Cede Latency: 10 Usage: 326573 Duration: 31098864616 -------------------------------------------------------------------------------- How to infer the results: Above Diff (%) and Below Diff (%) represent the number of cpuidle misses, indicating how frequently the selected cpuidle state was either too deep or too shallow. So, these values should not be too high. -------------------------------------------------------------------------------- The below test is done using a predictable timer and non-timer benchmark [1]: -------------------------------------------------------------------------------- Menu Governor: -------------------------------------------------------------------------------- With pipe wakeup (non-timer): -------------------------------------------------------------------------------------------------------- Sleep Interval Total Usage Diff Total Above Diff Above Diff (%) Total Below Diff Below Diff (%) -------------------------------------------------------------------------------------------------------- 0 5.980 1656304 2 0.000121 0 0.000000 1 10.959 901972 1954 0.216636 0 0.000000 -------------------------------------------------------------------------------------------------------- 2 15.726 243971 237112 97.188600 0 0.000000 3 20.813 232069 227258 97.926910 0 0.000000 4 30.896 209884 206492 98.383869 0 0.000000 5 40.991 216704 213642 98.587013 0 0.000000 6 51.002 195632 192963 98.635704 0 0.000000 7 61.014 163726 161506 98.644076 0 0.000000 8 71.006 140739 138809 98.628667 0 0.000000 9 81.008 123386 120725 97.843353 0 0.000000 10 101.020 98974 81235 82.077111 0 0.000000 -------------------------------------------------------------------------------------------------------- 11 111.044 90018 1513 1.680775 12 0.013331 12 121.015 82704 189 0.228526 77 0.093103 13 131.028 76534 272 0.355398 321 0.419421 14 141.008 71610 698 0.974724 693 0.967742 15 151.021 66869 666 0.995977 656 0.981023 16 161.027 62709 611 0.974342 605 0.964774 17 171.033 59063 593 1.004013 593 1.004013 18 181.019 55819 571 1.022949 541 0.969204 19 191.016 52998 641 1.209480 628 1.184950 20 201.017 50353 551 1.094274 501 0.994975 21 251.054 40535 289 0.712964 398 0.981868 22 301.037 33966 252 0.741918 330 0.971560 23 351.038 29279 216 0.737730 294 1.004133 24 401.047 25765 190 0.737435 262 1.016883 25 451.060 23021 185 0.803614 187 0.812302 26 501.049 20831 150 0.720081 216 1.036916 27 1001.076 10951 77 0.703132 126 1.150580 With timer wakeup: -------------------------------------------------------------------------------------------------------- Sleep Interval Total Usage Diff Total Above Diff Above Diff (%) Total Below Diff Below Diff (%) -------------------------------------------------------------------------------------------------------- 0 7.590 1310772 0 0.000000 0 0.000000 1 12.631 789377 780 0.098812 0 0.000000 2 21.791 458001 52321 11.423774 0 0.000000 3 22.648 440752 36 0.008168 0 0.000000 4 32.644 305983 0 0.000000 0 0.000000 5 42.646 234305 0 0.000000 0 0.000000 6 52.647 189858 2 0.001053 0 0.000000 7 62.649 159561 10 0.006267 0 0.000000 8 72.644 137643 5 0.003633 1 0.000727 9 82.666 120963 5 0.004133 0 0.000000 10 102.654 97442 3 0.003079 610 0.626013 11 145.805 69937 441 0.630568 1345 1.923159 12 156.057 64511 75 0.116259 439 0.680504 13 166.047 60765 215 0.353822 534 0.878795 14 175.894 57564 178 0.309221 687 1.193454 15 185.933 54471 255 0.468139 638 1.171265 16 195.975 51403 98 0.190650 212 0.412427 17 206.062 49281 174 0.353077 577 1.170837 18 216.188 46980 33 0.070243 571 1.215411 19 226.346 44879 30 0.066846 543 1.209920 20 236.353 43081 27 0.062673 516 1.197744 21 286.158 35782 5 0.013974 154 0.430384 22 336.730 30531 11 0.036029 266 0.871246 23 386.730 26722 18 0.067360 232 0.868198 24 436.770 23797 9 0.037820 196 0.823633 25 487.229 21359 13 0.060864 229 1.072148 26 537.375 19557 13 0.066472 259 1.324334 27 1037.871 10638 12 0.112803 127 1.193833 -------------------------------------------------------------------------------- Teo governor: -------------------------------------------------------------------------------- With pipe wakeup (non-timer): -------------------------------------------------------------------------------------------------------- Sleep Interval Total Usage Diff Total Above Diff Above Diff (%) Total Below Diff Below Diff (%) -------------------------------------------------------------------------------------------------------- 0 5.972 1657561 6 0.000362 0 0.000000 1 10.964 907279 0 0.000000 0 0.000000 2 15.977 623681 0 0.000000 0 0.000000 3 20.980 475385 1 0.000210 0 0.000000 4 30.981 322151 0 0.000000 0 0.000000 5 40.975 243749 0 0.000000 0 0.000000 6 50.977 195989 0 0.000000 0 0.000000 7 60.981 163876 0 0.000000 0 0.000000 8 70.978 140818 0 0.000000 0 0.000000 9 80.976 123460 0 0.000000 1 0.000810 10 100.970 99038 0 0.000000 16 0.016155 -------------------------------------------------------------------------------------------------------- 11 111.027 106919 5388 5.039329 16873 15.781105 -------------------------------------------------------------------------------------------------------- 12 121.017 83074 381 0.458627 443 0.533260 13 131.024 76588 323 0.421737 373 0.487021 14 141.027 71592 695 0.970779 695 0.970779 15 151.023 66874 669 1.000389 662 0.989921 16 161.026 62719 607 0.967809 595 0.948676 17 171.027 59064 581 0.983679 574 0.971827 18 181.020 55817 561 1.005070 532 0.953115 19 191.020 52883 523 0.988976 446 0.843371 20 201.025 50509 670 1.326496 645 1.277000 21 251.037 40544 280 0.690608 409 1.008781 22 301.033 33988 263 0.773803 349 1.026833 23 351.036 29285 221 0.754653 299 1.021001 24 401.042 25777 203 0.787524 272 1.055204 25 451.040 23029 174 0.755569 238 1.033480 26 501.048 20838 157 0.753431 224 1.074959 27 1001.097 10949 79 0.721527 134 1.223856 With timer wakeup: -------------------------------------------------------------------------------------------------------- Sleep Interval Total Usage Diff Total Above Diff Above Diff (%) Total Below Diff Below Diff (%) -------------------------------------------------------------------------------------------------------- 0 7.541 1319205 0 0.000000 0 0.000000 1 12.546 794464 0 0.000000 0 0.000000 2 17.540 568954 0 0.000000 0 0.000000 3 22.572 442307 0 0.000000 0 0.000000 4 32.583 306443 0 0.000000 1 0.000326 5 42.597 233238 0 0.000000 0 0.000000 6 52.587 190067 0 0.000000 0 0.000000 7 62.590 159714 0 0.000000 1 0.000626 8 72.574 137755 0 0.000000 2 0.001452 9 82.581 121081 0 0.000000 0 0.000000 10 102.589 97491 0 0.000000 1912 1.961207 11 146.385 68906 47 0.068209 599 0.869300 12 156.548 64565 86 0.133199 670 1.037714 13 166.588 60562 100 0.165120 518 0.855322 14 176.676 57264 263 0.459276 642 1.121123 15 186.563 54262 293 0.539973 601 1.107589 16 195.986 51668 192 0.371603 526 1.018038 17 206.860 49028 97 0.197846 564 1.150363 18 216.899 46669 27 0.057854 460 0.985665 19 227.016 44528 22 0.049407 367 0.824201 20 237.055 42883 28 0.065294 507 1.182287 21 286.998 35665 9 0.025235 283 0.793495 22 337.410 30439 7 0.022997 264 0.867308 23 387.522 26652 18 0.067537 251 0.941768 24 437.570 23742 8 0.033696 221 0.930840 25 487.804 21293 10 0.046964 94 0.441460 26 537.884 19505 7 0.035888 243 1.245834 27 1038.863 10633 4 0.037619 135 1.269632 -------------------------------------------------------------------------------- Teo Governor with patch -------------------------------------------------------------------------------- With pipe wakeup (non-timer): -------------------------------------------------------------------------------------------------------- Sleep Interval Total Usage Diff Total Above Diff Above Diff (%) Total Below Diff Below Diff (%) -------------------------------------------------------------------------------------------------------- 0 5.959 1661754 5 0.000301 0 0.000000 1 10.963 907497 2 0.000220 0 0.000000 2 15.968 623957 2 0.000321 0 0.000000 3 20.970 475574 2 0.000421 0 0.000000 4 30.974 321718 2 0.000622 0 0.000000 5 40.974 243714 2 0.000821 0 0.000000 6 50.983 195931 2 0.001021 0 0.000000 7 60.974 163876 2 0.001220 1 0.000610 8 70.973 140810 2 0.001420 1 0.000710 9 80.988 123420 1 0.000810 4 0.003241 10 100.994 99014 2 0.002020 20 0.020199 -------------------------------------------------------------------------------------------------------- 11 111.023 135597 11864 8.749456 45561 33.600301 => This is observed even -------------------------------------------------------------------------------------------------------- without the patch, 12 121.035 82948 348 0.419540 352 0.424362 when the sleep interval 13 131.019 76342 82 0.107411 48 0.062875 is almost equal to the 14 141.028 70948 70 0.098664 57 0.080341 residency time of state1. 15 151.023 66278 81 0.122212 69 0.104107 16 161.021 62146 65 0.104592 51 0.082065 17 171.023 58509 64 0.109385 47 0.080330 18 181.026 55301 64 0.115730 48 0.086798 19 191.033 52407 67 0.127846 45 0.085866 20 201.024 49803 52 0.104411 48 0.096380 21 251.042 39911 39 0.097717 45 0.112751 22 301.040 33302 29 0.087082 40 0.120113 23 351.045 28572 37 0.129497 34 0.118998 24 401.057 25005 20 0.079984 27 0.107978 25 451.055 22246 21 0.094399 26 0.116875 26 501.053 20031 14 0.069892 25 0.124807 27 1001.099 10055 7 0.069617 15 0.149180 With timer wakeup: -------------------------------------------------------------------------------------------------------- Sleep Interval Total Usage Diff Total Above Diff Above Diff (%) Total Below Diff Below Diff (%) -------------------------------------------------------------------------------------------------------- 0 7.566 1314872 0 0.000000 0 0.000000 1 12.553 794091 0 0.000000 0 0.000000 2 17.573 567627 0 0.000000 0 0.000000 3 22.631 441084 0 0.000000 0 0.000000 4 32.633 306095 0 0.000000 0 0.000000 5 42.631 234377 0 0.000000 0 0.000000 6 52.634 189899 0 0.000000 0 0.000000 7 62.642 159572 0 0.000000 0 0.000000 8 72.645 137619 0 0.000000 1 0.000727 9 82.616 121037 0 0.000000 1 0.000826 10 102.636 97423 0 0.000000 1047 1.074695 11 145.823 69245 43 0.062098 680 0.982020 12 155.912 64546 90 0.139435 478 0.740557 13 166.103 60709 219 0.360737 501 0.825248 14 176.036 57483 196 0.340970 639 1.111633 15 186.043 54448 250 0.459154 623 1.144211 16 195.552 51428 124 0.241114 230 0.447227 17 205.990 49270 141 0.286178 527 1.069616 18 216.300 46931 39 0.083101 545 1.161279 19 226.288 44884 23 0.051243 520 1.158542 20 236.372 43056 20 0.046451 493 1.145020 21 285.985 35724 1 0.002799 119 0.333109 22 336.636 30526 11 0.036035 262 0.858285 23 386.522 26772 17 0.063499 218 0.814284 24 436.749 23857 26 0.108983 208 0.871862 25 487.204 21358 9 0.042139 240 1.123701 26 537.312 19530 7 0.035842 235 1.203277 27 1038.147 10610 2 0.018850 119 1.121583 -------------------------------------------------------------------------------- I also did some tests with postgres (pgbench) - with the patch: +---------------------------+--------------------+------------------+ | Metric | Shared Cede | Snooze | +---------------------------+--------------------+------------------+ | Total Usage Difference | 119,453 | 20,472,846 | | Total Time Difference | 502.79 seconds | 324.90 seconds | | Total Above Difference | 74,500 (0.36%) | 0 | | Total Below Difference | 0 | 336,703 (1.64%) | +---------------------------+--------------------+------------------+ % Above Diff = 0.36% of total usage % Below Diff = 1.64% of total usage Using both the deterministic micro-benchmark and pgbench, I observed that the teo governor with the patch as anticipated does not cause any noticable increase in the cpuidle state prediction miss on PowerPC (pseries) (% above and below diff). So, for the entire series: Tested-by: Aboorva Devarajan <aboorvad@linux.ibm.com> [1]: https://github.com/AboorvaDevarajan/linux-utils/tree/main/cpuidle/cpuidle_wakeup
On Wed, Jan 15, 2025 at 3:52 PM Christian Loehle <christian.loehle@arm.com> wrote: > > On 1/13/25 18:32, Rafael J. Wysocki wrote: > > Hi Everyone, > > > > This supersedes > > > > https://lore.kernel.org/linux-pm/4953183.GXAFRqVoOG@rjwysocki.net/ > > > > but because the majority of patches in it are new, I've decided to count > > version numbers back from 1. > > > > This addresses a relatively recently added inconsistency in behavior of the teo > > governor regarding the handling of very frequent wakeups handling (patch [7/9]) > > and makes some other changes that may be regarded as cleanups. > > > > Please review. > > Hi Rafael, > that looks promising. I'll review the individual patches in detail now, but > I let a few tests run overnight and can report that there's no significant > behaviour change with the series on an arm64 (no polling state, rk3399), which > is my expected result. > > I'll get something running on a system with a polling state as well. > (I don't have a POWER system, so that will just be x86, adding Aboorva.) > > Tested-by: Christian Loehle <christian.loehle@arm.com> > for the entire series Thanks, much appreciated! > I'll just dump some of the raw results here for reference. > teo-X is teo with everything up to X/9 applied. teo-m is mainline, teo is > the entire series (equivalent to teo-9). > > Everything is done with fio direct random4k read. Range of real devices > with varying latency, mapper/slow is 51ms timer delay, nullb0 is > essentially CPU-bound, few intercepts expected. > -- > > device gov iter iops idles idle_misses idle_miss_ratio belows aboves > mmcblk1 menu 0 1284 344126 106309 0.309 89264 17045 > mmcblk1 menu 1 1340 358646 109469 0.305 91726 17743 > mmcblk1 menu 2 1316 348812 106773 0.306 89845 16928 > mmcblk1 teo 0 2023 559551 30053 0.054 2962 27091 > mmcblk1 teo 1 2007 548422 25816 0.047 2113 23703 > mmcblk1 teo 2 2010 557120 29434 0.053 2782 26652 > mmcblk1 teo-m 0 2010 542790 23336 0.043 1720 21616 > mmcblk1 teo-m 1 2009 554912 29239 0.053 2750 26489 > mmcblk1 teo-m 2 2013 548384 26444 0.048 2180 24264 > mmcblk1 teo-1 0 2030 524056 3571 0.007 610 2961 > mmcblk1 teo-1 1 2022 559304 27074 0.048 3373 23701 > mmcblk1 teo-1 2 2009 554744 27406 0.049 2964 24442 > mmcblk1 teo-2 0 2012 542886 18233 0.034 2140 16093 > mmcblk1 teo-2 1 2010 554094 27680 0.050 2957 24723 > mmcblk1 teo-2 2 2010 555222 28549 0.051 3085 25464 > mmcblk1 teo-3 0 2011 556816 28708 0.052 3076 25632 > mmcblk1 teo-3 1 2015 546998 20280 0.037 2414 17866 > mmcblk1 teo-3 2 2020 549758 20802 0.038 2662 18140 > mmcblk1 teo-4 0 2026 536464 11692 0.022 1457 10235 > mmcblk1 teo-4 1 2031 557978 25464 0.046 2928 22536 > mmcblk1 teo-4 2 2012 553812 26323 0.048 2943 23380 > mmcblk1 teo-5 0 2021 557966 3454 0.006 3003 451 > mmcblk1 teo-5 1 2027 546630 18400 0.034 2303 16097 > mmcblk1 teo-5 2 2012 554390 28013 0.051 2990 25023 > mmcblk1 teo-6 0 2037 539956 12764 0.024 1701 11063 > mmcblk1 teo-6 1 2019 556202 29074 0.052 3071 26003 > mmcblk1 teo-6 2 2017 555004 26814 0.048 2869 23945 > mmcblk1 teo-7 0 2028 545802 23806 0.044 1625 22181 > mmcblk1 teo-7 1 2019 557030 29913 0.054 2990 26923 > mmcblk1 teo-7 2 2018 553236 28534 0.052 2657 25877 > mmcblk1 teo-8 0 2018 554662 28595 0.052 2660 25935 > mmcblk1 teo-8 1 2014 554646 28947 0.052 2721 26226 > mmcblk1 teo-8 2 2014 543828 29379 0.054 2915 26464 > mmcblk1 teo-9 0 2013 555356 28867 0.052 2719 26148 > mmcblk1 teo-9 1 2015 557254 29709 0.053 2836 26873 > mmcblk1 teo-9 2 2030 559272 29321 0.052 3052 26269 > mmcblk2 menu 0 2885 434990 140864 0.324 123507 17357 > mmcblk2 menu 1 2766 450918 152931 0.339 135384 17547 > mmcblk2 menu 2 2960 407712 125573 0.308 109224 16349 > mmcblk2 teo 0 5677 741006 24131 0.033 2233 21898 > mmcblk2 teo 1 5652 714494 24412 0.034 2680 21732 > mmcblk2 teo 2 5660 752410 20497 0.027 1809 18688 > mmcblk2 teo-m 0 5683 855232 23454 0.027 1934 21520 > mmcblk2 teo-m 1 5588 517234 32016 0.062 5443 26573 > mmcblk2 teo-m 2 5654 668648 29108 0.044 3171 25937 > mmcblk2 teo-1 0 5666 773452 25740 0.033 3430 22310 > mmcblk2 teo-1 1 5638 668190 27619 0.041 4150 23469 > mmcblk2 teo-1 2 5689 866340 28191 0.033 3122 25069 > mmcblk2 teo-2 0 5769 859462 2116 0.002 1835 281 > mmcblk2 teo-2 1 5577 611956 25490 0.042 8647 16843 > mmcblk2 teo-2 2 5665 721308 25826 0.036 2876 22950 > mmcblk2 teo-3 0 5677 805946 30345 0.038 3877 26468 > mmcblk2 teo-3 1 5752 855840 14392 0.017 2588 11804 > mmcblk2 teo-3 2 5740 825580 27403 0.033 3750 23653 > mmcblk2 teo-4 0 5664 777848 3435 0.004 3229 206 > mmcblk2 teo-4 1 5660 728926 25796 0.035 2871 22925 > mmcblk2 teo-4 2 5689 774342 7844 0.010 2838 5006 > mmcblk2 teo-5 0 5672 781618 2733 0.003 2551 182 > mmcblk2 teo-5 1 5693 866530 28658 0.033 3332 25326 > mmcblk2 teo-5 2 5687 837294 12172 0.015 2338 9834 > mmcblk2 teo-6 0 5859 892798 5226 0.006 3566 1660 > mmcblk2 teo-6 1 5696 864197 24962 0.029 2796 22166 > mmcblk2 teo-6 2 5671 788200 26612 0.034 3157 23455 > mmcblk2 teo-7 0 5617 525864 30143 0.057 3739 26404 > mmcblk2 teo-7 1 5732 802794 25553 0.032 3240 22313 > mmcblk2 teo-7 2 5838 858620 12661 0.015 188 12473 > mmcblk2 teo-8 0 5696 780894 24440 0.031 2928 21512 > mmcblk2 teo-8 1 5868 862794 12978 0.015 187 12791 > mmcblk2 teo-8 2 5607 660124 31632 0.048 6728 24904 > mmcblk2 teo-9 0 5652 669318 28330 0.042 2692 25638 > mmcblk2 teo-9 1 5657 710892 29291 0.041 3233 26058 > mmcblk2 teo-9 2 5594 740988 37086 0.050 10275 26811 > nvme0n1 menu 0 4775 384186 70675 0.184 54346 16329 > nvme0n1 menu 1 5211 409374 69545 0.170 53484 16061 > nvme0n1 menu 2 7213 535088 81312 0.152 64253 17059 > nvme0n1 teo 0 10655 754882 29476 0.039 2960 26516 > nvme0n1 teo 1 10627 755322 29324 0.039 3011 26313 > nvme0n1 teo 2 10559 749498 29850 0.040 3265 26585 > nvme0n1 teo-m 0 10353 738802 29937 0.041 2996 26941 > nvme0n1 teo-m 1 10629 748148 27076 0.036 2349 24727 > nvme0n1 teo-m 2 10477 743256 28193 0.038 2645 25548 > nvme0n1 teo-1 0 10947 772905 24896 0.032 3150 21746 > nvme0n1 teo-1 1 10412 732894 21868 0.030 2423 19445 > nvme0n1 teo-1 2 10530 748377 26338 0.035 3479 22859 > nvme0n1 teo-2 0 10570 751961 5195 0.007 2853 2342 > nvme0n1 teo-2 1 10482 732428 18667 0.025 2102 16565 > nvme0n1 teo-2 2 10829 768292 26400 0.034 3189 23211 > nvme0n1 teo-3 0 10493 746638 26371 0.035 3320 23051 > nvme0n1 teo-3 1 10445 742924 27871 0.038 3027 24844 > nvme0n1 teo-3 2 10920 775112 25646 0.033 3520 22126 > nvme0n1 teo-4 0 10500 734792 18125 0.025 2283 15842 > nvme0n1 teo-4 1 11091 783828 25904 0.033 3031 22873 > nvme0n1 teo-4 2 10413 736248 23786 0.032 2660 21126 > nvme0n1 teo-5 0 10479 744340 26933 0.036 3096 23837 > nvme0n1 teo-5 1 10740 764188 27065 0.035 3062 24003 > nvme0n1 teo-5 2 10500 747060 25962 0.035 3233 22729 > nvme0n1 teo-6 0 10715 757772 2186 0.003 1872 314 > nvme0n1 teo-6 1 10485 734878 17620 0.024 2174 15446 > nvme0n1 teo-6 2 10478 744106 26909 0.036 3007 23902 > nvme0n1 teo-7 0 10549 750022 30416 0.041 3320 27096 > nvme0n1 teo-7 1 10611 752182 29877 0.040 3332 26545 > nvme0n1 teo-7 2 11170 791572 30790 0.039 3453 27337 > nvme0n1 teo-8 0 10622 752523 29002 0.039 2745 26257 > nvme0n1 teo-8 1 10424 739904 28641 0.039 2756 25885 > nvme0n1 teo-8 2 10533 731440 22101 0.030 1280 20821 > nvme0n1 teo-9 0 10367 727768 24895 0.034 1820 23075 > nvme0n1 teo-9 1 10815 766230 29378 0.038 3136 26242 > nvme0n1 teo-9 2 10426 739224 28173 0.038 2563 25610 > sda menu 0 872 526240 175645 0.334 159373 16272 > sda menu 1 900 536434 188122 0.351 170749 17373 > sda menu 2 901 534826 189778 0.355 171745 18033 > sda teo 0 1687 999108 16015 0.016 1249 14766 > sda teo 1 1668 1007551 31316 0.031 5809 25507 > sda teo 2 1682 985830 18784 0.019 1003 17781 > sda teo-m 0 1676 969984 25825 0.027 3191 22634 > sda teo-m 1 1682 995266 30340 0.030 4040 26300 > sda teo-m 2 1681 940562 20830 0.022 1508 19322 > sda teo-1 0 1680 996214 15553 0.016 2192 13361 > sda teo-1 1 1674 1050168 28033 0.027 4011 24022 > sda teo-1 2 1665 808202 35292 0.044 8873 26419 > sda teo-2 0 1689 1013360 546 0.001 243 303 > sda teo-2 1 1672 1033806 29432 0.028 4363 25069 > sda teo-2 2 1667 1046100 31110 0.030 6395 24715 > sda teo-3 0 1625 922694 40891 0.044 22925 17966 > sda teo-3 1 1670 894480 27946 0.031 3045 24901 > sda teo-3 2 1658 1009366 28514 0.028 8887 19627 > sda teo-4 0 1674 977280 3605 0.004 3279 326 > sda teo-4 1 1677 960990 2861 0.003 1058 1803 > sda teo-4 2 1660 1016592 33894 0.033 7687 26207 > sda teo-5 0 1678 1033470 16996 0.016 2163 14833 > sda teo-5 1 1667 873077 25011 0.029 4393 20618 > sda teo-5 2 1672 1042020 24589 0.024 3858 20731 > sda teo-6 0 1680 962686 7255 0.008 1797 5458 > sda teo-6 1 1682 1055472 28602 0.027 5081 23521 > sda teo-6 2 1675 989244 19110 0.019 5620 13490 > sda teo-7 0 1618 1001000 51178 0.051 26011 25167 > sda teo-7 1 1666 1047310 34032 0.032 6520 27512 > sda teo-7 2 1676 982046 28788 0.029 2836 25952 > sda teo-8 0 1618 975930 52929 0.054 26386 26543 > sda teo-8 1 1678 1023796 16182 0.016 2191 13991 > sda teo-8 2 1633 981693 43520 0.044 18782 24738 > sda teo-9 0 1661 1029494 32423 0.031 5855 26568 > sda teo-9 1 1678 969400 17843 0.018 2763 15080 > sda teo-9 2 1679 1055060 32288 0.031 5607 26681 > mtdblock3 menu 0 180 278442 80547 0.289 65714 14833 > mtdblock3 menu 1 167 379840 109168 0.287 93460 15708 > mtdblock3 menu 2 155 373948 126262 0.338 107799 18463 > mtdblock3 teo 0 256 663190 29333 0.044 4836 24497 > mtdblock3 teo 1 257 808114 29394 0.036 2742 26652 > mtdblock3 teo 2 257 456150 25812 0.057 2468 23344 > mtdblock3 teo-m 0 257 624492 24678 0.040 2326 22352 > mtdblock3 teo-m 1 256 734548 31165 0.042 4307 26858 > mtdblock3 teo-m 2 257 812004 30479 0.038 3510 26969 > mtdblock3 teo-1 0 253 759708 34146 0.045 10081 24065 > mtdblock3 teo-1 1 254 730160 23105 0.032 7156 15949 > mtdblock3 teo-1 2 253 721558 32848 0.046 10028 22820 > mtdblock3 teo-2 0 257 666426 4948 0.007 782 4166 > mtdblock3 teo-2 1 256 682046 25126 0.037 4886 20240 > mtdblock3 teo-2 2 253 639950 36029 0.056 11074 24955 > mtdblock3 teo-3 0 252 709122 37198 0.052 12603 24595 > mtdblock3 teo-3 1 257 709680 30000 0.042 3670 26330 > mtdblock3 teo-3 2 254 540408 32696 0.061 9035 23661 > mtdblock3 teo-4 0 256 442356 28281 0.064 3785 24496 > mtdblock3 teo-4 1 254 588362 16934 0.029 5663 11271 > mtdblock3 teo-4 2 257 628776 28300 0.045 3667 24633 > mtdblock3 teo-5 0 250 762594 43008 0.056 16752 26256 > mtdblock3 teo-5 1 256 586098 29744 0.051 4035 25709 > mtdblock3 teo-5 2 262 903736 375 0.000 139 236 > mtdblock3 teo-6 0 250 795274 42742 0.054 17160 25582 > mtdblock3 teo-6 1 256 800172 28195 0.035 4016 24179 > mtdblock3 teo-6 2 257 523365 26260 0.050 2788 23472 > mtdblock3 teo-7 0 261 944626 31616 0.033 5090 26526 > mtdblock3 teo-7 1 258 647048 28997 0.045 2728 26269 > mtdblock3 teo-7 2 260 901858 18521 0.021 3698 14823 > mtdblock3 teo-8 0 256 914076 27272 0.030 6844 20428 > mtdblock3 teo-8 1 256 809113 31696 0.039 4672 27024 > mtdblock3 teo-8 2 252 798396 40223 0.050 13270 26953 > mtdblock3 teo-9 0 259 726886 30515 0.042 4002 26513 > mtdblock3 teo-9 1 262 892854 9411 0.011 1686 7725 > mtdblock3 teo-9 2 259 909528 16583 0.018 3054 13529 > nullb0 menu 0 107364 86086 20668 0.240 2604 18064 > nullb0 menu 1 107701 85072 20251 0.238 2808 17443 > nullb0 menu 2 107976 85178 20591 0.242 3063 17528 > nullb0 teo 0 105758 84464 28872 0.342 3429 25443 > nullb0 teo 1 106892 87122 30255 0.347 3722 26533 > nullb0 teo 2 107080 83374 27911 0.335 2918 24993 > nullb0 teo-m 0 107702 88337 29824 0.338 2956 26868 > nullb0 teo-m 1 108218 88130 29710 0.337 3038 26672 > nullb0 teo-m 2 106993 86866 30033 0.346 3664 26369 > nullb0 teo-1 0 106416 82936 23635 0.285 3186 20449 > nullb0 teo-1 1 107570 87425 27682 0.317 3220 24462 > nullb0 teo-1 2 107832 75656 19434 0.257 2775 16659 > nullb0 teo-2 0 106320 88182 28603 0.324 3213 25390 > nullb0 teo-2 1 107316 79968 21643 0.271 3081 18562 > nullb0 teo-2 2 108007 84018 25244 0.300 3034 22210 > nullb0 teo-3 0 106989 85160 24558 0.288 3601 20957 > nullb0 teo-3 1 107325 83582 24994 0.299 2963 22031 > nullb0 teo-3 2 107063 77066 18978 0.246 2913 16065 > nullb0 teo-4 0 107990 86156 25012 0.290 3404 21608 > nullb0 teo-4 1 107352 75750 15096 0.199 2671 12425 > nullb0 teo-4 2 107685 87644 26047 0.297 3825 22222 > nullb0 teo-5 0 106687 87788 13428 0.153 2937 10491 > nullb0 teo-5 1 107820 86442 25703 0.297 3686 22017 > nullb0 teo-5 2 108319 81498 22895 0.281 2713 20182 > nullb0 teo-6 0 107784 84364 24239 0.287 3459 20780 > nullb0 teo-6 1 108762 83854 24017 0.286 3289 20728 > nullb0 teo-6 2 106024 88238 27778 0.315 3253 24525 > nullb0 teo-7 0 106041 86354 30076 0.348 3420 26656 > nullb0 teo-7 1 108039 79564 26805 0.337 2924 23881 > nullb0 teo-7 2 107389 88264 30713 0.348 3156 27557 > nullb0 teo-8 0 106465 87924 29679 0.338 2958 26721 > nullb0 teo-8 1 107215 81348 27038 0.332 2916 24122 > nullb0 teo-8 2 108238 87070 29284 0.336 2971 26313 > nullb0 teo-9 0 107220 84508 28894 0.342 3411 25483 > nullb0 teo-9 1 107400 86920 29945 0.345 3579 26366 > nullb0 teo-9 2 107845 87686 30439 0.347 3784 26655 > mapper/dm-slow menu 0 19 85216 20060 0.235 2478 17582 > mapper/dm-slow teo 0 19 86454 28959 0.335 2860 26099 > mapper/dm-slow teo-m 0 19 84740 28056 0.331 2582 25474 > mapper/dm-slow teo-1 0 19 88698 28945 0.326 3210 25735 > mapper/dm-slow teo-2 0 19 83138 24326 0.293 2589 21737 > mapper/dm-slow teo-3 0 19 86416 27274 0.316 2998 24276 > mapper/dm-slow teo-4 0 19 89286 29369 0.329 3151 26218 > mapper/dm-slow teo-5 0 19 88516 29213 0.330 3163 26050 > mapper/dm-slow teo-6 0 19 87896 24675 0.281 3113 21562 > mapper/dm-slow teo-7 0 19 83986 27976 0.333 2788 25188 > mapper/dm-slow teo-8 0 19 88072 29667 0.337 2962 26705 > mapper/dm-slow teo-9 0 19 87260 29428 0.337 2923 26505 >
On Mon, Jan 20, 2025 at 9:17 AM Aboorva Devarajan <aboorvad@linux.ibm.com> wrote: > > On Wed, 2025-01-15 at 14:52 +0000, Christian Loehle wrote: > > On 1/13/25 18:32, Rafael J. Wysocki wrote: > > > Hi Everyone, > > > > > > This supersedes > > > > > > https://lore.kernel.org/linux-pm/4953183.GXAFRqVoOG@rjwysocki.net/ > > > > > > but because the majority of patches in it are new, I've decided to count > > > version numbers back from 1. > > > > > > This addresses a relatively recently added inconsistency in behavior of the teo > > > governor regarding the handling of very frequent wakeups handling (patch [7/9]) > > > and makes some other changes that may be regarded as cleanups. > > > > > > Please review. > > > > Hi Rafael, > > that looks promising. I'll review the individual patches in detail now, but > > I let a few tests run overnight and can report that there's no significant > > behaviour change with the series on an arm64 (no polling state, rk3399), which > > is my expected result. > > > > I'll get something running on a system with a polling state as well. > > (I don't have a POWER system, so that will just be x86, adding Aboorva.) > > > Christian, > > Thanks for adding me to the thread. > > Rafael, > > I did some tests with the patch on a Pseries Power10 system: > > Here are the system details: > > -------------------------------------------------------------------------------- > Architecture: ppc64le > Byte Order: Little Endian > CPU(s): 48 > On-line CPU(s) list: 0-47 > Model name: POWER10 (architected), altivec supported > Model: 2.0 (pvr 0080 0200) > Thread(s) per core: 8 > Core(s) per socket: 6 > Socket(s): 1 > Physical sockets: 8 > Physical chips: 1 > Physical cores/chip: 10 > > -------------------------------------------------------------------------------- > # cpupower idle-info > CPUidle driver: pseries_idle > CPUidle governor: menu > analyzing CPU 5: > > Number of idle states: 2 > Available idle states: snooze Shared Cede > snooze: > Flags/Description: snooze > Latency: 0 > Usage: 1411724 > Duration: 27481504 > Shared Cede: > Flags/Description: Shared Cede > Latency: 10 > Usage: 326573 > Duration: 31098864616 > -------------------------------------------------------------------------------- > > How to infer the results: > > Above Diff (%) and Below Diff (%) represent the number of cpuidle misses, > indicating how frequently the selected cpuidle state was either too deep or > too shallow. So, these values should not be too high. > -------------------------------------------------------------------------------- > > The below test is done using a predictable timer and non-timer benchmark [1]: > > -------------------------------------------------------------------------------- > Menu Governor: > -------------------------------------------------------------------------------- > > With pipe wakeup (non-timer): > > -------------------------------------------------------------------------------------------------------- > Sleep Interval Total Usage Diff Total Above Diff Above Diff (%) Total Below Diff Below Diff (%) > -------------------------------------------------------------------------------------------------------- > 0 5.980 1656304 2 0.000121 0 0.000000 > 1 10.959 901972 1954 0.216636 0 0.000000 > -------------------------------------------------------------------------------------------------------- > 2 15.726 243971 237112 97.188600 0 0.000000 > 3 20.813 232069 227258 97.926910 0 0.000000 > 4 30.896 209884 206492 98.383869 0 0.000000 > 5 40.991 216704 213642 98.587013 0 0.000000 > 6 51.002 195632 192963 98.635704 0 0.000000 > 7 61.014 163726 161506 98.644076 0 0.000000 > 8 71.006 140739 138809 98.628667 0 0.000000 > 9 81.008 123386 120725 97.843353 0 0.000000 > 10 101.020 98974 81235 82.077111 0 0.000000 > -------------------------------------------------------------------------------------------------------- > 11 111.044 90018 1513 1.680775 12 0.013331 > 12 121.015 82704 189 0.228526 77 0.093103 > 13 131.028 76534 272 0.355398 321 0.419421 > 14 141.008 71610 698 0.974724 693 0.967742 > 15 151.021 66869 666 0.995977 656 0.981023 > 16 161.027 62709 611 0.974342 605 0.964774 > 17 171.033 59063 593 1.004013 593 1.004013 > 18 181.019 55819 571 1.022949 541 0.969204 > 19 191.016 52998 641 1.209480 628 1.184950 > 20 201.017 50353 551 1.094274 501 0.994975 > 21 251.054 40535 289 0.712964 398 0.981868 > 22 301.037 33966 252 0.741918 330 0.971560 > 23 351.038 29279 216 0.737730 294 1.004133 > 24 401.047 25765 190 0.737435 262 1.016883 > 25 451.060 23021 185 0.803614 187 0.812302 > 26 501.049 20831 150 0.720081 216 1.036916 > 27 1001.076 10951 77 0.703132 126 1.150580 > > With timer wakeup: > > -------------------------------------------------------------------------------------------------------- > Sleep Interval Total Usage Diff Total Above Diff Above Diff (%) Total Below Diff Below Diff (%) > -------------------------------------------------------------------------------------------------------- > 0 7.590 1310772 0 0.000000 0 0.000000 > 1 12.631 789377 780 0.098812 0 0.000000 > 2 21.791 458001 52321 11.423774 0 0.000000 > 3 22.648 440752 36 0.008168 0 0.000000 > 4 32.644 305983 0 0.000000 0 0.000000 > 5 42.646 234305 0 0.000000 0 0.000000 > 6 52.647 189858 2 0.001053 0 0.000000 > 7 62.649 159561 10 0.006267 0 0.000000 > 8 72.644 137643 5 0.003633 1 0.000727 > 9 82.666 120963 5 0.004133 0 0.000000 > 10 102.654 97442 3 0.003079 610 0.626013 > 11 145.805 69937 441 0.630568 1345 1.923159 > 12 156.057 64511 75 0.116259 439 0.680504 > 13 166.047 60765 215 0.353822 534 0.878795 > 14 175.894 57564 178 0.309221 687 1.193454 > 15 185.933 54471 255 0.468139 638 1.171265 > 16 195.975 51403 98 0.190650 212 0.412427 > 17 206.062 49281 174 0.353077 577 1.170837 > 18 216.188 46980 33 0.070243 571 1.215411 > 19 226.346 44879 30 0.066846 543 1.209920 > 20 236.353 43081 27 0.062673 516 1.197744 > 21 286.158 35782 5 0.013974 154 0.430384 > 22 336.730 30531 11 0.036029 266 0.871246 > 23 386.730 26722 18 0.067360 232 0.868198 > 24 436.770 23797 9 0.037820 196 0.823633 > 25 487.229 21359 13 0.060864 229 1.072148 > 26 537.375 19557 13 0.066472 259 1.324334 > 27 1037.871 10638 12 0.112803 127 1.193833 > > -------------------------------------------------------------------------------- > Teo governor: > -------------------------------------------------------------------------------- > > With pipe wakeup (non-timer): > > -------------------------------------------------------------------------------------------------------- > Sleep Interval Total Usage Diff Total Above Diff Above Diff (%) Total Below Diff Below Diff (%) > -------------------------------------------------------------------------------------------------------- > 0 5.972 1657561 6 0.000362 0 0.000000 > 1 10.964 907279 0 0.000000 0 0.000000 > 2 15.977 623681 0 0.000000 0 0.000000 > 3 20.980 475385 1 0.000210 0 0.000000 > 4 30.981 322151 0 0.000000 0 0.000000 > 5 40.975 243749 0 0.000000 0 0.000000 > 6 50.977 195989 0 0.000000 0 0.000000 > 7 60.981 163876 0 0.000000 0 0.000000 > 8 70.978 140818 0 0.000000 0 0.000000 > 9 80.976 123460 0 0.000000 1 0.000810 > 10 100.970 99038 0 0.000000 16 0.016155 > -------------------------------------------------------------------------------------------------------- > 11 111.027 106919 5388 5.039329 16873 15.781105 > -------------------------------------------------------------------------------------------------------- > 12 121.017 83074 381 0.458627 443 0.533260 > 13 131.024 76588 323 0.421737 373 0.487021 > 14 141.027 71592 695 0.970779 695 0.970779 > 15 151.023 66874 669 1.000389 662 0.989921 > 16 161.026 62719 607 0.967809 595 0.948676 > 17 171.027 59064 581 0.983679 574 0.971827 > 18 181.020 55817 561 1.005070 532 0.953115 > 19 191.020 52883 523 0.988976 446 0.843371 > 20 201.025 50509 670 1.326496 645 1.277000 > 21 251.037 40544 280 0.690608 409 1.008781 > 22 301.033 33988 263 0.773803 349 1.026833 > 23 351.036 29285 221 0.754653 299 1.021001 > 24 401.042 25777 203 0.787524 272 1.055204 > 25 451.040 23029 174 0.755569 238 1.033480 > 26 501.048 20838 157 0.753431 224 1.074959 > 27 1001.097 10949 79 0.721527 134 1.223856 > > With timer wakeup: > > -------------------------------------------------------------------------------------------------------- > Sleep Interval Total Usage Diff Total Above Diff Above Diff (%) Total Below Diff Below Diff (%) > -------------------------------------------------------------------------------------------------------- > 0 7.541 1319205 0 0.000000 0 0.000000 > 1 12.546 794464 0 0.000000 0 0.000000 > 2 17.540 568954 0 0.000000 0 0.000000 > 3 22.572 442307 0 0.000000 0 0.000000 > 4 32.583 306443 0 0.000000 1 0.000326 > 5 42.597 233238 0 0.000000 0 0.000000 > 6 52.587 190067 0 0.000000 0 0.000000 > 7 62.590 159714 0 0.000000 1 0.000626 > 8 72.574 137755 0 0.000000 2 0.001452 > 9 82.581 121081 0 0.000000 0 0.000000 > 10 102.589 97491 0 0.000000 1912 1.961207 > 11 146.385 68906 47 0.068209 599 0.869300 > 12 156.548 64565 86 0.133199 670 1.037714 > 13 166.588 60562 100 0.165120 518 0.855322 > 14 176.676 57264 263 0.459276 642 1.121123 > 15 186.563 54262 293 0.539973 601 1.107589 > 16 195.986 51668 192 0.371603 526 1.018038 > 17 206.860 49028 97 0.197846 564 1.150363 > 18 216.899 46669 27 0.057854 460 0.985665 > 19 227.016 44528 22 0.049407 367 0.824201 > 20 237.055 42883 28 0.065294 507 1.182287 > 21 286.998 35665 9 0.025235 283 0.793495 > 22 337.410 30439 7 0.022997 264 0.867308 > 23 387.522 26652 18 0.067537 251 0.941768 > 24 437.570 23742 8 0.033696 221 0.930840 > 25 487.804 21293 10 0.046964 94 0.441460 > 26 537.884 19505 7 0.035888 243 1.245834 > 27 1038.863 10633 4 0.037619 135 1.269632 > > -------------------------------------------------------------------------------- > Teo Governor with patch > -------------------------------------------------------------------------------- > > With pipe wakeup (non-timer): > > -------------------------------------------------------------------------------------------------------- > Sleep Interval Total Usage Diff Total Above Diff Above Diff (%) Total Below Diff Below Diff (%) > -------------------------------------------------------------------------------------------------------- > 0 5.959 1661754 5 0.000301 0 0.000000 > 1 10.963 907497 2 0.000220 0 0.000000 > 2 15.968 623957 2 0.000321 0 0.000000 > 3 20.970 475574 2 0.000421 0 0.000000 > 4 30.974 321718 2 0.000622 0 0.000000 > 5 40.974 243714 2 0.000821 0 0.000000 > 6 50.983 195931 2 0.001021 0 0.000000 > 7 60.974 163876 2 0.001220 1 0.000610 > 8 70.973 140810 2 0.001420 1 0.000710 > 9 80.988 123420 1 0.000810 4 0.003241 > 10 100.994 99014 2 0.002020 20 0.020199 > -------------------------------------------------------------------------------------------------------- > 11 111.023 135597 11864 8.749456 45561 33.600301 => This is observed even > -------------------------------------------------------------------------------------------------------- without the patch, > 12 121.035 82948 348 0.419540 352 0.424362 when the sleep interval > 13 131.019 76342 82 0.107411 48 0.062875 is almost equal to the > 14 141.028 70948 70 0.098664 57 0.080341 residency time of state1. > 15 151.023 66278 81 0.122212 69 0.104107 > 16 161.021 62146 65 0.104592 51 0.082065 > 17 171.023 58509 64 0.109385 47 0.080330 > 18 181.026 55301 64 0.115730 48 0.086798 > 19 191.033 52407 67 0.127846 45 0.085866 > 20 201.024 49803 52 0.104411 48 0.096380 > 21 251.042 39911 39 0.097717 45 0.112751 > 22 301.040 33302 29 0.087082 40 0.120113 > 23 351.045 28572 37 0.129497 34 0.118998 > 24 401.057 25005 20 0.079984 27 0.107978 > 25 451.055 22246 21 0.094399 26 0.116875 > 26 501.053 20031 14 0.069892 25 0.124807 > 27 1001.099 10055 7 0.069617 15 0.149180 > > > With timer wakeup: > > -------------------------------------------------------------------------------------------------------- > Sleep Interval Total Usage Diff Total Above Diff Above Diff (%) Total Below Diff Below Diff (%) > -------------------------------------------------------------------------------------------------------- > 0 7.566 1314872 0 0.000000 0 0.000000 > 1 12.553 794091 0 0.000000 0 0.000000 > 2 17.573 567627 0 0.000000 0 0.000000 > 3 22.631 441084 0 0.000000 0 0.000000 > 4 32.633 306095 0 0.000000 0 0.000000 > 5 42.631 234377 0 0.000000 0 0.000000 > 6 52.634 189899 0 0.000000 0 0.000000 > 7 62.642 159572 0 0.000000 0 0.000000 > 8 72.645 137619 0 0.000000 1 0.000727 > 9 82.616 121037 0 0.000000 1 0.000826 > 10 102.636 97423 0 0.000000 1047 1.074695 > 11 145.823 69245 43 0.062098 680 0.982020 > 12 155.912 64546 90 0.139435 478 0.740557 > 13 166.103 60709 219 0.360737 501 0.825248 > 14 176.036 57483 196 0.340970 639 1.111633 > 15 186.043 54448 250 0.459154 623 1.144211 > 16 195.552 51428 124 0.241114 230 0.447227 > 17 205.990 49270 141 0.286178 527 1.069616 > 18 216.300 46931 39 0.083101 545 1.161279 > 19 226.288 44884 23 0.051243 520 1.158542 > 20 236.372 43056 20 0.046451 493 1.145020 > 21 285.985 35724 1 0.002799 119 0.333109 > 22 336.636 30526 11 0.036035 262 0.858285 > 23 386.522 26772 17 0.063499 218 0.814284 > 24 436.749 23857 26 0.108983 208 0.871862 > 25 487.204 21358 9 0.042139 240 1.123701 > 26 537.312 19530 7 0.035842 235 1.203277 > 27 1038.147 10610 2 0.018850 119 1.121583 > -------------------------------------------------------------------------------- > > > I also did some tests with postgres (pgbench) - with the patch: > > +---------------------------+--------------------+------------------+ > | Metric | Shared Cede | Snooze | > +---------------------------+--------------------+------------------+ > | Total Usage Difference | 119,453 | 20,472,846 | > | Total Time Difference | 502.79 seconds | 324.90 seconds | > | Total Above Difference | 74,500 (0.36%) | 0 | > | Total Below Difference | 0 | 336,703 (1.64%) | > +---------------------------+--------------------+------------------+ > > % Above Diff = 0.36% of total usage > % Below Diff = 1.64% of total usage > > > Using both the deterministic micro-benchmark and pgbench, I observed that the > teo governor with the patch as anticipated does not cause any noticable increase in the cpuidle > state prediction miss on PowerPC (pseries) (% above and below diff). > > So, for the entire series: > > Tested-by: Aboorva Devarajan <aboorvad@linux.ibm.com> Thank you, much appreciated! > [1]: https://github.com/AboorvaDevarajan/linux-utils/tree/main/cpuidle/cpuidle_wakeup