From patchwork Tue Apr 19 16:14:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 563841 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13C8DC433F5 for ; Tue, 19 Apr 2022 16:15:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354486AbiDSQRz (ORCPT ); Tue, 19 Apr 2022 12:17:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52708 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1354482AbiDSQRw (ORCPT ); Tue, 19 Apr 2022 12:17:52 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 330D1B7F9 for ; Tue, 19 Apr 2022 09:15:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1650384907; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QMMLV+aQ8jiYUAcGfbODPuNoPXpnRo47CDI6W0o+6/8=; b=WXpnsrwflhGcDvdoVYELZF+Tb/F40b3DoxMftYQmlhWBtCY7kCWKPtmB8WtVLl0IudWtGJ O71jwMOm9+hIaSYu+C439a8gTTNVK2j1v92Kfi9pdAgjtDAiBdc+YWgH2WU8uvLBuOFjvU qHMK6d+02/BLnDgrQZYHqWLZND202dg= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-527-dWqOmsPWNaOYXjla-ikOPw-1; Tue, 19 Apr 2022 12:14:57 -0400 X-MC-Unique: dWqOmsPWNaOYXjla-ikOPw-1 Received: by mail-wm1-f72.google.com with SMTP id h65-20020a1c2144000000b0038e9ce3b29cso1560410wmh.2 for ; Tue, 19 Apr 2022 09:14:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QMMLV+aQ8jiYUAcGfbODPuNoPXpnRo47CDI6W0o+6/8=; b=ArVf1kR/pi4HFatPXawi1OCOgI8IVDgosDJdbyCUS2+tfcUaAQJ1uR5kbeHZ9bHu0t Arua+blrSJ6hUbmyKbYjLGLIqzpJXxNt6MCc4ZWVzaDt61J3jBB0hFtUxZyrS4Q+WSzK VCmBTqiaySQlA+GyOLJgEB/YTFs+FIgxFk1U8oRP/JiAdN7q5KnM0hL26gbwkPeVObBM 2THhJ4K6uWM8l7uGv+H9JOOPK92E3RFU315l9erPC76e5R0awa/yRLbFoJfy+02owIqd lc88HjDcSZFUZtDh4mGTMjwJh2p0VB7gj+8uwErryNNCG+5jdhHKIhnlpTDy9cyGWQCJ WJYw== X-Gm-Message-State: AOAM533B7huT6jMPT320Tm9zklCGFHAeCFNBIbX4dtOlCAq6At82et5l 793ihQsoNzGYp3PtE3ZP7JzH6kH59d94cjO7rZBkgtKNQxnrvlM3H1zHIC3bY+L0mUG7oTy6qHI uStw5okyl4cQu1s0pyT9yg6x3/YPVVFDAy3srXAN2cDFJ+HUZJbzPYwp2DCDBVW5deWrsnTdV5Z jQzEE= X-Received: by 2002:a05:600c:4f53:b0:392:e99:3002 with SMTP id m19-20020a05600c4f5300b003920e993002mr16844475wmq.35.1650384896257; Tue, 19 Apr 2022 09:14:56 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzCM0WEFGlMdjdBiXiBEEc5owN2kPNoXXL23G6VrMBtdOQo8nvEqlPdHvA6QmZ1uHh+NGz6lw== X-Received: by 2002:a05:600c:4f53:b0:392:e99:3002 with SMTP id m19-20020a05600c4f5300b003920e993002mr16844458wmq.35.1650384896032; Tue, 19 Apr 2022 09:14:56 -0700 (PDT) Received: from vschneid.remote.csb ([185.11.37.247]) by smtp.gmail.com with ESMTPSA id b13-20020a05600018ad00b0020a87feadfcsm8682381wri.84.2022.04.19.09.14.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Apr 2022 09:14:55 -0700 (PDT) From: Valentin Schneider To: linux-rt-users@vger.kernel.org Cc: John Kacur , Clark Williams Subject: [PATCH 1/3] rteval: systopology: Fix offline NUMA node parsing Date: Tue, 19 Apr 2022 17:14:41 +0100 Message-Id: <20220419161443.89674-2-vschneid@redhat.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220419161443.89674-1-vschneid@redhat.com> References: <20220419161443.89674-1-vschneid@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org An offline NUMA node will report in its cpulist an empty string. Unfortunately, "".split(sep=x) with x != None returns a list containing an empty string rather than an empty list, which causes CpuList._expand_cpulist() to try to run int(''), which ends up in the following exception: ValueError: invalid literal for int() with base 10: '' Prevent this by adding an early empty-string check. Signed-off-by: Valentin Schneider Signed-off-by: John Kacur --- rteval/systopology.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/rteval/systopology.py b/rteval/systopology.py index bf794ce..b2da7bb 100644 --- a/rteval/systopology.py +++ b/rteval/systopology.py @@ -103,6 +103,10 @@ class CpuList: don't error check against online cpus """ result = [] + + if not cpulist: + return result + for part in cpulist.split(','): if '-' in part: a, b = part.split('-') From patchwork Tue Apr 19 16:14:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 563842 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F4CAC433F5 for ; Tue, 19 Apr 2022 16:15:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354483AbiDSQRo (ORCPT ); Tue, 19 Apr 2022 12:17:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52184 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238184AbiDSQRn (ORCPT ); Tue, 19 Apr 2022 12:17:43 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id B23D6BC21 for ; Tue, 19 Apr 2022 09:15:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1650384899; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DcRv8Pq2Wi5OQorGor9al/2Mb6MqsLk1kf7iSmdEiF4=; b=HyIZBj2oDNwH0C/hrJ0Z590qh7j7bV9XCaG+sMXaZ5SQJPq96ajaWMg79LuBcEfxmMSlrG bwCt4+9ltjvWD3WHdNdKWFz8vSNPx20pwjVrM+8ClmVrJC/ZZHMxtcMIzoCxXeG8UmUdDo gtByahZPtdNmAoh2zRp4zYiLG1yJ+9Y= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-571-VyMxjECzNpq6KqmALagFTA-1; Tue, 19 Apr 2022 12:14:58 -0400 X-MC-Unique: VyMxjECzNpq6KqmALagFTA-1 Received: by mail-wm1-f72.google.com with SMTP id k16-20020a7bc310000000b0038e6cf00439so1449932wmj.0 for ; Tue, 19 Apr 2022 09:14:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DcRv8Pq2Wi5OQorGor9al/2Mb6MqsLk1kf7iSmdEiF4=; b=GkX07usu0qTWzibJMpUFNCg4ZXfq+3z6k+r0wRJpfzZTGxzRZyOVxt1pBJHvcBloXy Uni6xZ5zvBAkbiIbYRr/tOkVFeYHauRsbbXlw8JLlP2bvIKwfQd6fHmzBJs5zaga2CrC CN+ZP+2/KgPxdCmAQUzOggmYKY9Os2Vn+ONNANziFeDgYN3jWmFTwAwYfVHM9uONVKdO aK94iI/VKaJpvmiDU4vKonIzl74IoeSikEjaLAVsq8tjIXXKPK8rWOyHiP6qH5HO7XWl 5F7bzFIK0AKFhTD86Fh3nAYOH/K0vqrgqvloaKn4kQ5Mv+T+FAi9ixEmRvfXNdUD5cJF kOzA== X-Gm-Message-State: AOAM533fm7KOkW12/S3G7gf75I6vh+BY7UpFyDxXwG/YP3pHy/4bzDoz brODjHR8laOE+BB7plNJyr8E7mVGNKSO7WRTLyNMOqP1LYor+Nr/AK37/xjPeeKtuqHcAui5JPc 37fsvYfwTMk/KLLySn6jlPCcO30+tN1dz//7MKP0P39gQZ4F8DBoXYnLHXyJzAP9s0tnoedbu9n MkkNo= X-Received: by 2002:a05:600c:35c3:b0:392:33c1:65b1 with SMTP id r3-20020a05600c35c300b0039233c165b1mr17964563wmq.86.1650384897232; Tue, 19 Apr 2022 09:14:57 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyu1ihzbVjYpMhtJZBoFbkNerKzGLfigt5QlNopRyxyM4kbvAt3+6KRQWNE+Jnv4uShQUSt1Q== X-Received: by 2002:a05:600c:35c3:b0:392:33c1:65b1 with SMTP id r3-20020a05600c35c300b0039233c165b1mr17964549wmq.86.1650384897053; Tue, 19 Apr 2022 09:14:57 -0700 (PDT) Received: from vschneid.remote.csb ([185.11.37.247]) by smtp.gmail.com with ESMTPSA id b13-20020a05600018ad00b0020a87feadfcsm8682381wri.84.2022.04.19.09.14.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Apr 2022 09:14:56 -0700 (PDT) From: Valentin Schneider To: linux-rt-users@vger.kernel.org Cc: John Kacur , Clark Williams Subject: [PATCH 2/3] rteval: kcompile: Fix offline node handling Date: Tue, 19 Apr 2022 17:14:42 +0100 Message-Id: <20220419161443.89674-3-vschneid@redhat.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220419161443.89674-1-vschneid@redhat.com> References: <20220419161443.89674-1-vschneid@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org Having an empty NumaNode but with CPUs attached to it (IOW they are all offline) causes kcompile.py to raise the following exception: calc_jobs_per_cpu(): ratio = float(mem) / float(len(self.node)) ZeroDivisionError: float division by zero Remove nodes that do have CPUs but none of which are online. Signed-off-by: Valentin Schneider --- rteval/modules/loads/kcompile.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/rteval/modules/loads/kcompile.py b/rteval/modules/loads/kcompile.py index 367f8dc..ac99964 100644 --- a/rteval/modules/loads/kcompile.py +++ b/rteval/modules/loads/kcompile.py @@ -211,7 +211,10 @@ class Kcompile(CommandLineLoad): # remove nodes with no cpus available for running for node, cpus in self.cpus.items(): - if not cpus: + # If the intersection between the node CPUs and the cpulist is empty + # then either the cpulist exludes that node, or the CPUs allowed by + # the cpulist are actually offline + if not set(self.topology.nodes[node].cpus.cpulist) & set(cpus): self.nodes.remove(node) self._log(Log.DEBUG, "node %s has no available cpus, removing" % node) From patchwork Tue Apr 19 16:14:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 564708 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E627C433EF for ; Tue, 19 Apr 2022 16:15:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354474AbiDSQRq (ORCPT ); Tue, 19 Apr 2022 12:17:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52298 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1354482AbiDSQRo (ORCPT ); Tue, 19 Apr 2022 12:17:44 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id BC62CB7F9 for ; Tue, 19 Apr 2022 09:15:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1650384900; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aOdBf52A4yq3oIDmLPsLpVZzeAmgmmyrVQYlhaKxLdI=; b=Xlp+3Qfb8fecOCYagbN2JGWAadbmSeXIc1UspJbNFD4t/Ea8jfgadGyEamICPuJOZ1wiDM +YBkIcSvCNfLW0Fw2aadGrGCw9WdGRB/krKnbLXlKUsAuO5orT/7qsr8EKoDc9ySC0/I5L RLjtZpTkGrWEMrNK1gof6MToKPviaDY= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-601-5ra9LXJaNrS-eNrsISTIIQ-1; Tue, 19 Apr 2022 12:14:59 -0400 X-MC-Unique: 5ra9LXJaNrS-eNrsISTIIQ-1 Received: by mail-wm1-f70.google.com with SMTP id c62-20020a1c3541000000b0038ec265155fso1550759wma.6 for ; Tue, 19 Apr 2022 09:14:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aOdBf52A4yq3oIDmLPsLpVZzeAmgmmyrVQYlhaKxLdI=; b=W0+LuRosR9hFcuoH8YmqVuxksn8RBXbF6f26lDyYH4No16VvYKQVf3PLVpERJmoBDc q/SL/Qja5CMGdNB1gCPLErbTON2olRm3DvKOpIU3gAygXR+oiqOLRC7AnJyy9GUnkWTU 9yjGeWQLZbNR9TpANyIjWqpAOit4Z0jNzn/GmjdWqbiQPRhwAZjpRRw1iAWfDtQqGRgK D48iJGtdpokgBxD5BcSIhUNSLbvHtOtMfY5miOnuDk8JJyJML4kYhgBhwbBivpZxv5CM eFmH6DyJaEAL0RGZDFW7qe+Rk4VkaSTW44vHeHLABwrcLiNM35izw9ifoe4KRE9SHlVB Rr6Q== X-Gm-Message-State: AOAM531dIW145aICDlwZ+htNKiiNzqei5XpfPF9tzD+vridWWHWv/qf8 4u6C4tNwuZXMqwwEvLwZTSSqvlTdDSkg0vgEHlQM1h68nCblVcbaJxN5upp2FMDN+Ur5WXsnDZx AS4tVnoRpLd4diCx1E/Ovt25N6mtlMaE5LfgGej5PHGoZ5TsXjxVqwsHRQD/++ropn2pyAOv7si 1jd14= X-Received: by 2002:a5d:5955:0:b0:207:8444:203b with SMTP id e21-20020a5d5955000000b002078444203bmr12227689wri.433.1650384898172; Tue, 19 Apr 2022 09:14:58 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwpEZQaTWb0S+s9osED8NGL2BjD8O0xr+iG1nJT6DCnKqW5mlxwgxQIBHhTw6cgx4E7o6lXRg== X-Received: by 2002:a5d:5955:0:b0:207:8444:203b with SMTP id e21-20020a5d5955000000b002078444203bmr12227672wri.433.1650384897947; Tue, 19 Apr 2022 09:14:57 -0700 (PDT) Received: from vschneid.remote.csb ([185.11.37.247]) by smtp.gmail.com with ESMTPSA id b13-20020a05600018ad00b0020a87feadfcsm8682381wri.84.2022.04.19.09.14.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Apr 2022 09:14:57 -0700 (PDT) From: Valentin Schneider To: linux-rt-users@vger.kernel.org Cc: John Kacur , Clark Williams Subject: [PATCH 3/3] rteval: systopology: Slight CpuList.__expand_cpulist() cleanup Date: Tue, 19 Apr 2022 17:14:43 +0100 Message-Id: <20220419161443.89674-4-vschneid@redhat.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220419161443.89674-1-vschneid@redhat.com> References: <20220419161443.89674-1-vschneid@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org This method currently aggregates CPUs into a list, then converts this to set and then back to list. The aggregation can instead be done directly into a set. (as an offside, it would make more sense for CpuList to have its storage be a set in the first place as duplicate CPU ids don't make sense for it, but that's a separate discussion :-)) The integer conversion of the "a-b" pattern can also be condensed into a single map() expression. Signed-off-by: Valentin Schneider --- rteval/systopology.py | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/rteval/systopology.py b/rteval/systopology.py index b2da7bb..2a28f9c 100644 --- a/rteval/systopology.py +++ b/rteval/systopology.py @@ -102,20 +102,19 @@ class CpuList: """ expand a range string into an array of cpu numbers don't error check against online cpus """ - result = [] - if not cpulist: - return result + return [] + + result = set() for part in cpulist.split(','): if '-' in part: - a, b = part.split('-') - a, b = int(a), int(b) - result.extend(list(range(a, b + 1))) + a, b = map(int, part.split('-')) + result |= set(range(a, b + 1)) else: a = int(part) - result.append(a) - return [int(i) for i in list(set(result))] + result |= {a} + return list(result) def getcpulist(self): """ return the list of cpus tracked """