From patchwork Fri Aug 14 17:34:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Herring X-Patchwork-Id: 247739 Delivered-To: patch@linaro.org Received: by 2002:a92:cc90:0:0:0:0:0 with SMTP id x16csp554143ilo; Fri, 14 Aug 2020 10:34:27 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw94tWmKGzGy60Cts9s/OUsexXnZVsucUoFeca/qjpkVh4yOfrjtVoX1dvks32vJVYvzwx8 X-Received: by 2002:a17:906:957:: with SMTP id j23mr3431069ejd.344.1597426467661; Fri, 14 Aug 2020 10:34:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1597426467; cv=none; d=google.com; s=arc-20160816; b=g2erkFJf8f90Vksi4lsYmVS4WA7EoibTf+GHI1stGy2sJC/hvZah/JDfQ2nyv8lgVq skKC0AHAfA5NNxlU8yOTLLSZsXve/3Co23V3Uh/hVlsCkmwRdaaAQO4s0OPszwUo0Ltm 1GvjZFNqcx4ZHL36zED1x/QGSaVmvFkE5zbKF7ii6p8sm3TUf/BNqpfTpOFFmbh6sYBa F7qzYVHCuW6BGCB9FL1nz/sBM341xx/VESSnjHxM5nrnW0J9j7fZ011FA983Y+ubE39O XyguUuSuqQqHSCYD5EU75kLr/re1MCfZ2wRI629kxWYXwh5OuYASqhyvL4a6BWaA53Xo /IgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=3l69ln56x0NYyOcK12UG1i2L2ctArBHsZcQkVLntWAw=; b=Y7AezlkTbn6iH1BiXCsbljveFogcKrZRBY//6Hw8qetRgfhXT+cJG3BQYb+qx4N7dG g8I4h4e+fliuF0Ee+PpgRmpJ2cwywQmDaD1mfbdSAA6Q8FNo+6hVbUjiUE+Z2M8BJJkK jNXQbyN3NfDhw42R+IWY1C7im0YJnHToVDly4ESVzWshI1/JOYVFzhknWvdbC2ggCRBM ii/DB3S8GW9cKKlwaYxGy7wb+bhjg+fibJ+MM/MdmUdjkb9I5NDk0vr46EhpRSRkTvgj eBx0pSoNOXbAo52nAVbVmic+siKR/8MwtE6NA6/lnZKMQQTxLTxGjKYQyrSqzDRFOdDN +VQQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=devicetree-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id b10si5714877eje.310.2020.08.14.10.34.27; Fri, 14 Aug 2020 10:34:27 -0700 (PDT) Received-SPF: pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=devicetree-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728767AbgHNRe0 (ORCPT + 6 others); Fri, 14 Aug 2020 13:34:26 -0400 Received: from mail-il1-f194.google.com ([209.85.166.194]:35118 "EHLO mail-il1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728431AbgHNReV (ORCPT ); Fri, 14 Aug 2020 13:34:21 -0400 Received: by mail-il1-f194.google.com with SMTP id q14so5670246ilm.2; Fri, 14 Aug 2020 10:34:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3l69ln56x0NYyOcK12UG1i2L2ctArBHsZcQkVLntWAw=; b=JvZNKUX7ajrYA2652rREivlZ8aaAX3pVh7wumpmDnTZb40OA7x8phwhSmG4eeSnTia gZCr0hUWwNQuLCXgq2110bz0t213BGc1QD44r8vUh0I4+hyLm0XhmbpERI8e8r1xJF5x XlCxaczngeGZoziPqtYs5Sph/uy7veLEN9xXQwQ5lwrIBcJ7IALpMupV0JeHyRqpvWfH 8YyUP4FH8ssudqW4vICc/gftQtApg4hAtCRHVatQipcp763TXtNfp0owMKSQCPmNNft+ LrarsoZkmfEc7V5+eOMQX0JZNg+GnKoCd/tJiv/lHIEMFAuzaFXUt7j6KtyaGdWFATx7 Y7DA== X-Gm-Message-State: AOAM532m3RX7ZCFuGd2xh4DMdG0Jdnp7DZuUMd7QI1FmBGSa7PFH0K+p Nf9rGAAbh/ujFF5pzm6i2TGWu7fRpw== X-Received: by 2002:a92:874a:: with SMTP id d10mr3423940ilm.273.1597426460524; Fri, 14 Aug 2020 10:34:20 -0700 (PDT) Received: from xps15.herring.priv ([64.188.179.249]) by smtp.googlemail.com with ESMTPSA id x185sm4637575iof.41.2020.08.14.10.34.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Aug 2020 10:34:20 -0700 (PDT) From: Rob Herring To: devicetree@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Andrei Ziureaev , Masahiro Yamada Subject: [PATCH 1/3] dt-bindings: Bump minimum version of dtschema to 2020.8 Date: Fri, 14 Aug 2020 11:34:15 -0600 Message-Id: <20200814173417.2322509-2-robh@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200814173417.2322509-1-robh@kernel.org> References: <20200814173417.2322509-1-robh@kernel.org> MIME-Version: 1.0 Sender: devicetree-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org dtschema release 2020.8 gained several additions to help performance. dt-doc-validate can now take a list of files and directories, and dt-mk-schema can store the processed schema in JSON which is much faster to parse than YAML. Utilizing both of these changes results in a 4-5x speed improvement in running dt_binding_check. There's also additional meta-schema checks which binding schemas should be checked against. Signed-off-by: Rob Herring --- Documentation/devicetree/bindings/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 2.25.1 diff --git a/Documentation/devicetree/bindings/Makefile b/Documentation/devicetree/bindings/Makefile index 91c4d00e96d3..6a678eb5b5cd 100644 --- a/Documentation/devicetree/bindings/Makefile +++ b/Documentation/devicetree/bindings/Makefile @@ -3,7 +3,7 @@ DT_DOC_CHECKER ?= dt-doc-validate DT_EXTRACT_EX ?= dt-extract-example DT_MK_SCHEMA ?= dt-mk-schema -DT_SCHEMA_MIN_VERSION = 2020.5 +DT_SCHEMA_MIN_VERSION = 2020.8 PHONY += check_dtschema_version check_dtschema_version: From patchwork Fri Aug 14 17:34:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Herring X-Patchwork-Id: 247741 Delivered-To: patch@linaro.org Received: by 2002:a92:cc90:0:0:0:0:0 with SMTP id x16csp554232ilo; Fri, 14 Aug 2020 10:34:35 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxsAXB/UzBhl67pbZ2Q0aXeBkUJ1xqtLi5z6ouywBKuyuGJZP5hu1imDHqKoaU+cEBkOMas X-Received: by 2002:a17:906:cb8c:: with SMTP id mf12mr3450324ejb.3.1597426475141; Fri, 14 Aug 2020 10:34:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1597426475; cv=none; d=google.com; s=arc-20160816; b=GW6Bw8xaJgltX3R2MOrn4Wz0C57oASvmmwcKFcQDup35ZdqsqVN1gpB+yr+VmQxrPO ml2fC8tOakZ9t5RTj7uzZqcK9j/gNLbQlS8Wg9/5FgHCOgu/JtUkn0g+PYk+njuWK/ME 7yIkmhasjktE/C/RSBgNkTgQpMnTLe3kiGu/puP8wHyWHJzSM1vrjUkKu7FBi/3ZiXDx ja1r8HSv8mOBgVk4Hl+Oe3d4llOJ9H2Mi1BEN9A20LSqYuxi3YVEMkb6L0nw152YJjzl pyuxql+zhwZtPfc0afQpYeKifSTz2hEGbMo60K5PPUGc3pcIKJNu0VrTuSRk5L5O03JR a9VQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=6vBukXRpzXGxxaRWK2ZPECr+qES1cRfcn/6HodQxij4=; b=FzVWPINHFVEYAsqv0vTO3wZh1x6xnX4MDjbxTSgDV4G34LBVuBHFLX9+l/2VCbxdq8 KJLpzlzUQA7RRuDbE/Pj5IW7/UPflAgib1QEdNoTpK4/FbhKDvao7xgi2QnvbQyr7LoF 00tG1yndIb/4hiaKg3icVwRQp1iCJqxpuhX/JtGMUXxGzMQJopGJWmM1noNG3bv9TiCu UGhq2Ou6fEw47j5j3xB2TPwoPxPn6lFPsPTI4Xj5Dkox/nRfpwdqUfSzB+WrKamlf9jJ qjIS1uVN9PFVmmwhXY4Jz6BlwIBzA3fq0sqz3UhLPOcysoBnBf7yeKKyCVaAGWd/H3Z4 ZMnw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=devicetree-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id sd25si5746527ejb.341.2020.08.14.10.34.34; Fri, 14 Aug 2020 10:34:35 -0700 (PDT) Received-SPF: pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=devicetree-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728830AbgHNRee (ORCPT + 6 others); Fri, 14 Aug 2020 13:34:34 -0400 Received: from mail-io1-f68.google.com ([209.85.166.68]:33734 "EHLO mail-io1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728765AbgHNReX (ORCPT ); Fri, 14 Aug 2020 13:34:23 -0400 Received: by mail-io1-f68.google.com with SMTP id g14so11627873iom.0; Fri, 14 Aug 2020 10:34:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6vBukXRpzXGxxaRWK2ZPECr+qES1cRfcn/6HodQxij4=; b=ZlHBsS56oorpgPD696c5Jzw6EWMZDje4ZsA37e3slXf8S8LOmb0gi+KjBP9aHyC9gv KvRKxppPIeD7ECfPNx9IfRjs4lXpot+orX2SBpXury1kYs8JMm0SxQ84PG0dxgZ9KgMG k78RDtUwWyKaTbPGR9dgjUCTNung0SqfoSa19apQN9QlW/FQ8n2/Ek+B2AKJhfBtfag3 aXdRM+GL4xbdIHPGCPOXKZq4Ckh1yK5TjvlPenjnGStFgu+upHvxn1exYtdzJh3Xf1ij ADkekhQV45iJ5EmwBa83JN/APC5yGo0bAGdO2rWAsCuIzNU/ZT5grDZmKZW6Ra1lEtMF +dhw== X-Gm-Message-State: AOAM530J5TGeslul+8RbEfwy/1Bo0sURjkWsIxgqheA6rYUhjrLcKMsc rJ6PDbCT7jcI27dEGrKe6Z4NWl09Hw== X-Received: by 2002:a02:a04d:: with SMTP id f13mr3672108jah.112.1597426462005; Fri, 14 Aug 2020 10:34:22 -0700 (PDT) Received: from xps15.herring.priv ([64.188.179.249]) by smtp.googlemail.com with ESMTPSA id x185sm4637575iof.41.2020.08.14.10.34.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Aug 2020 10:34:21 -0700 (PDT) From: Rob Herring To: devicetree@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Andrei Ziureaev , Masahiro Yamada Subject: [PATCH 2/3] dt-bindings: Use json for processed-schema* Date: Fri, 14 Aug 2020 11:34:16 -0600 Message-Id: <20200814173417.2322509-3-robh@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200814173417.2322509-1-robh@kernel.org> References: <20200814173417.2322509-1-robh@kernel.org> MIME-Version: 1.0 Sender: devicetree-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org From: Andrei Ziureaev Change the format of processed-schema* from yaml to json to speed up validation. With json output, using xargs and appending the output won't work since json has explicit list begin and end characters. Instead, we pass the schema files as a list in a temp file. The parsing time for the processed schema goes down from ~2sec to 70ms. Also, 'make dtbs_check' becomes 33% faster. Some error messages are affected by this change. For example, "True was expected" becomes "... is not of type 'boolean'". The order of messages is also changed. Signed-off-by: Andrei Ziureaev Signed-off-by: Rob Herring --- Documentation/devicetree/bindings/.gitignore | 1 + Documentation/devicetree/bindings/Makefile | 25 ++++++++++---------- scripts/Makefile.lib | 2 +- 3 files changed, 15 insertions(+), 13 deletions(-) -- 2.25.1 diff --git a/Documentation/devicetree/bindings/.gitignore b/Documentation/devicetree/bindings/.gitignore index 5c6d8ea1a09c..3a05b99bfa26 100644 --- a/Documentation/devicetree/bindings/.gitignore +++ b/Documentation/devicetree/bindings/.gitignore @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only *.example.dts processed-schema*.yaml +processed-schema*.json diff --git a/Documentation/devicetree/bindings/Makefile b/Documentation/devicetree/bindings/Makefile index 6a678eb5b5cd..bedc2210a2e0 100644 --- a/Documentation/devicetree/bindings/Makefile +++ b/Documentation/devicetree/bindings/Makefile @@ -19,18 +19,19 @@ $(obj)/%.example.dts: $(src)/%.yaml check_dtschema_version FORCE $(call if_changed,chk_binding) # Use full schemas when checking %.example.dts -DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml +DT_TMP_SCHEMA := $(obj)/processed-schema-examples.json find_cmd = find $(srctree)/$(src) \( -name '*.yaml' ! \ -name 'processed-schema*' ! \ -name '*.example.dt.yaml' \) quiet_cmd_mk_schema = SCHEMA $@ - cmd_mk_schema = rm -f $@ ; \ + cmd_mk_schema = f=$$(mktemp) ; \ $(if $(DT_MK_SCHEMA_FLAGS), \ echo $(real-prereqs), \ - $(find_cmd)) | \ - xargs $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) >> $@ + $(find_cmd)) > $$f ; \ + $(DT_MK_SCHEMA) -j $(DT_MK_SCHEMA_FLAGS) @$$f > $@ ; \ + rm -f $$f DT_DOCS = $(shell $(find_cmd) | sed -e 's|^$(srctree)/||') @@ -39,33 +40,33 @@ override DTC_FLAGS := \ -Wno-graph_child_address \ -Wno-interrupt_provider -$(obj)/processed-schema-examples.yaml: $(DT_DOCS) check_dtschema_version FORCE +$(obj)/processed-schema-examples.json: $(DT_DOCS) check_dtschema_version FORCE $(call if_changed,mk_schema) ifeq ($(DT_SCHEMA_FILES),) # Unless DT_SCHEMA_FILES is specified, use the full schema for dtbs_check too. -# Just copy processed-schema-examples.yaml +# Just copy processed-schema-examples.json -$(obj)/processed-schema.yaml: $(obj)/processed-schema-examples.yaml FORCE +$(obj)/processed-schema.json: $(obj)/processed-schema-examples.json FORCE $(call if_changed,copy) DT_SCHEMA_FILES = $(DT_DOCS) else -# If DT_SCHEMA_FILES is specified, use it for processed-schema.yaml +# If DT_SCHEMA_FILES is specified, use it for processed-schema.json -$(obj)/processed-schema.yaml: DT_MK_SCHEMA_FLAGS := -u -$(obj)/processed-schema.yaml: $(DT_SCHEMA_FILES) check_dtschema_version FORCE +$(obj)/processed-schema.json: DT_MK_SCHEMA_FLAGS := -u +$(obj)/processed-schema.json: $(DT_SCHEMA_FILES) check_dtschema_version FORCE $(call if_changed,mk_schema) endif extra-$(CHECK_DT_BINDING) += $(patsubst $(src)/%.yaml,%.example.dts, $(DT_SCHEMA_FILES)) extra-$(CHECK_DT_BINDING) += $(patsubst $(src)/%.yaml,%.example.dt.yaml, $(DT_SCHEMA_FILES)) -extra-$(CHECK_DT_BINDING) += processed-schema-examples.yaml -extra-$(CHECK_DTBS) += processed-schema.yaml +extra-$(CHECK_DT_BINDING) += processed-schema-examples.json +extra-$(CHECK_DTBS) += processed-schema.json # Hack: avoid 'Argument list too long' error for 'make clean'. Remove most of # build artifacts here before they are processed by scripts/Makefile.clean diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 3d599716940c..94133708889d 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -328,7 +328,7 @@ $(obj)/%.dtb: $(src)/%.dts $(DTC) FORCE DT_CHECKER ?= dt-validate DT_BINDING_DIR := Documentation/devicetree/bindings # DT_TMP_SCHEMA may be overridden from Documentation/devicetree/bindings/Makefile -DT_TMP_SCHEMA ?= $(objtree)/$(DT_BINDING_DIR)/processed-schema.yaml +DT_TMP_SCHEMA ?= $(objtree)/$(DT_BINDING_DIR)/processed-schema.json quiet_cmd_dtb_check = CHECK $@ cmd_dtb_check = $(DT_CHECKER) -u $(srctree)/$(DT_BINDING_DIR) -p $(DT_TMP_SCHEMA) $@ From patchwork Fri Aug 14 17:34:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Herring X-Patchwork-Id: 247740 Delivered-To: patch@linaro.org Received: by 2002:a92:cc90:0:0:0:0:0 with SMTP id x16csp554224ilo; Fri, 14 Aug 2020 10:34:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyxtETbfCB6go9rdg/emcVzxMG98g6Mx2AfXvAefvsKFKnFvjTszGkDeIxnT8DDre2VyIQX X-Received: by 2002:a17:906:496:: with SMTP id f22mr3405108eja.180.1597426474714; Fri, 14 Aug 2020 10:34:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1597426474; cv=none; d=google.com; s=arc-20160816; b=xaumlMWiP0rTumbLGGcmdQYiAlFz0H/YJqbcywwWkvbZFU8M5EkzE9niqzoQjg9q4Z l6VUw2RjKUE6fJUVmXLQMdEYX+AeiJwxVAVH96FNbwuiAGIocOpg9m393adPt6gh8BAg 8Cb9JYwG6CjqkQGRH5Km/cPVz+P1iSLlWMZJNV7Yjlk7XSXtuBWGkZUwi5WlUALErCmZ 7bUpvO/NIe08CN6TiXDFw/XmsD0GubogXar/sUhBnJ7BivKaKGy7GhSxnN22ALVHsMu1 VQaq15Y5TWyJ7+5cMKkmG+30eUIYGUQ24ZxbIAgfhdVJ2YXMQ2bkbpeeRA9TyHe0F9dY 1Xuw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=OBzjqYjSkJEbEHMceXH+VuiJYK8gX6oWBcQRGJhBqfE=; b=EACggoufZRdnB4+c8XqHSb/wXTWqH9/YKu5IC3H4lNm0BmpJCo61Kbw47XwHMDEXe7 3wwrdKJ++AJfRjSzLZkRf5BuPD0rDP+Jc4xvUYf3levUyjMd2rcoElLSJRL7Wm1BLHjp VL3LN2yrvyHmbREQmWwRCPCMfYipE6PNjRPJXN3gMnukFxsHUmVx60tmUaGvhF4wMw/G xYBZji30R9V4c/DBIQEPgiYUVXeHBZvrPhUxCkCUZAf2kOprGzxJLwWc9pBH8gLYX5rh HuyvvHqegyzjBxD30HtlOK2K8VBPBco0SkAvDwenuspqtGMJEwpqtYdaNN3bZEDyLbWC agZA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=devicetree-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id sd25si5746527ejb.341.2020.08.14.10.34.34; Fri, 14 Aug 2020 10:34:34 -0700 (PDT) Received-SPF: pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=devicetree-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728431AbgHNRed (ORCPT + 6 others); Fri, 14 Aug 2020 13:34:33 -0400 Received: from mail-io1-f67.google.com ([209.85.166.67]:42603 "EHLO mail-io1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728419AbgHNReY (ORCPT ); Fri, 14 Aug 2020 13:34:24 -0400 Received: by mail-io1-f67.google.com with SMTP id j8so11488283ioe.9; Fri, 14 Aug 2020 10:34:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OBzjqYjSkJEbEHMceXH+VuiJYK8gX6oWBcQRGJhBqfE=; b=G8zedPQiWnLalFbPw+D7877COMUUyC+TdqeULlSD3+LkAiM/W3Vy1TBMgLXQmRFwWs /Tvy5QCM9zQ8zX3ZGugITcfu/+ZBQ06FCIIfu9mfIFTHFmwcFkr8J8Qwgvk6iiLFEw0H uHGC2oftBlFXwv8BsH4fLOOQgyc0KeNsE8BeBWeR7sEo+Abd2ocFd3FQ7Rr4VqbZWndk jls1mGwYII99aqXUAboa4cskARQQvwCXqbdnfkfZ1NbluQrJzTeGx2vbvDrs//me8GzE Yhs5ejOm0OlhRklOvzqxD1OPKipyNFuSsAI6xOYzzhWBH9/scbxlenw2K4xiDwuLWcr+ JPZw== X-Gm-Message-State: AOAM532BWXGBl0hK1StB+o2vDNICMhFzrPE/jgaZWACRn4oyMK9Z4BBj 9LEkfQLNzbZXNTo93Pi1EriR0mFS0w== X-Received: by 2002:a02:3e11:: with SMTP id s17mr3573532jas.67.1597426463407; Fri, 14 Aug 2020 10:34:23 -0700 (PDT) Received: from xps15.herring.priv ([64.188.179.249]) by smtp.googlemail.com with ESMTPSA id x185sm4637575iof.41.2020.08.14.10.34.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Aug 2020 10:34:22 -0700 (PDT) From: Rob Herring To: devicetree@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Andrei Ziureaev , Masahiro Yamada Subject: [PATCH 3/3] dt-bindings: Validate DT binding schema in a single call Date: Fri, 14 Aug 2020 11:34:17 -0600 Message-Id: <20200814173417.2322509-4-robh@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200814173417.2322509-1-robh@kernel.org> References: <20200814173417.2322509-1-robh@kernel.org> MIME-Version: 1.0 Sender: devicetree-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org As the number of binding schemas has grown, the time to run dt_binding_check has gotten pretty slow. A large part of this is due to the slow startup time of Python (a well documented problem). There's not currently any benefit to running dt-doc-validate one file at a time, so let's switch it to run a single rule. Doing this means we loose the make parallelism, but we can use xargs instead. This speeds up the validation time from several minutes to <10 sec. Since the validation is a single step with no output, we move running it as part of the processed-schema-examples.json target. We also need to reorder the extra-y entries so the validation is run first rather than after all the examples are extracted. Signed-off-by: Rob Herring --- Documentation/devicetree/bindings/Makefile | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) -- 2.25.1 diff --git a/Documentation/devicetree/bindings/Makefile b/Documentation/devicetree/bindings/Makefile index bedc2210a2e0..d21c4c4f9eee 100644 --- a/Documentation/devicetree/bindings/Makefile +++ b/Documentation/devicetree/bindings/Makefile @@ -11,12 +11,11 @@ check_dtschema_version: $(DT_DOC_CHECKER) --version 2>/dev/null || echo 0; } | sort -VC || \ { echo "ERROR: dtschema minimum version is v$(DT_SCHEMA_MIN_VERSION)" >&2; false; } -quiet_cmd_chk_binding = CHKDT $(patsubst $(srctree)/%,%,$<) - cmd_chk_binding = $(DT_DOC_CHECKER) -u $(srctree)/$(src) $< ; \ - $(DT_EXTRACT_EX) $< > $@ +quiet_cmd_extract_ex = DTEX $@ + cmd_extract_ex = $(DT_EXTRACT_EX) $< > $@ $(obj)/%.example.dts: $(src)/%.yaml check_dtschema_version FORCE - $(call if_changed,chk_binding) + $(call if_changed,extract_ex) # Use full schemas when checking %.example.dts DT_TMP_SCHEMA := $(obj)/processed-schema-examples.json @@ -25,6 +24,10 @@ find_cmd = find $(srctree)/$(src) \( -name '*.yaml' ! \ -name 'processed-schema*' ! \ -name '*.example.dt.yaml' \) +quiet_cmd_chk_bindings = CHKDT $@ + cmd_chk_bindings = $(find_cmd) | \ + xargs -n200 -P$$(nproc) $(DT_DOC_CHECKER) -u $(srctree)/$(src) + quiet_cmd_mk_schema = SCHEMA $@ cmd_mk_schema = f=$$(mktemp) ; \ $(if $(DT_MK_SCHEMA_FLAGS), \ @@ -33,6 +36,11 @@ quiet_cmd_mk_schema = SCHEMA $@ $(DT_MK_SCHEMA) -j $(DT_MK_SCHEMA_FLAGS) @$$f > $@ ; \ rm -f $$f +define rule_chkdt + $(call cmd,chk_bindings) + $(call cmd,mk_schema) +endef + DT_DOCS = $(shell $(find_cmd) | sed -e 's|^$(srctree)/||') override DTC_FLAGS := \ @@ -41,7 +49,7 @@ override DTC_FLAGS := \ -Wno-interrupt_provider $(obj)/processed-schema-examples.json: $(DT_DOCS) check_dtschema_version FORCE - $(call if_changed,mk_schema) + $(call if_changed_rule,chkdt) ifeq ($(DT_SCHEMA_FILES),) @@ -63,10 +71,10 @@ $(obj)/processed-schema.json: $(DT_SCHEMA_FILES) check_dtschema_version FORCE endif -extra-$(CHECK_DT_BINDING) += $(patsubst $(src)/%.yaml,%.example.dts, $(DT_SCHEMA_FILES)) -extra-$(CHECK_DT_BINDING) += $(patsubst $(src)/%.yaml,%.example.dt.yaml, $(DT_SCHEMA_FILES)) extra-$(CHECK_DT_BINDING) += processed-schema-examples.json extra-$(CHECK_DTBS) += processed-schema.json +extra-$(CHECK_DT_BINDING) += $(patsubst $(src)/%.yaml,%.example.dts, $(DT_SCHEMA_FILES)) +extra-$(CHECK_DT_BINDING) += $(patsubst $(src)/%.yaml,%.example.dt.yaml, $(DT_SCHEMA_FILES)) # Hack: avoid 'Argument list too long' error for 'make clean'. Remove most of # build artifacts here before they are processed by scripts/Makefile.clean