From patchwork Thu Feb 6 17:30:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Maydell X-Patchwork-Id: 183128 Delivered-To: patch@linaro.org Received: by 2002:a92:1f12:0:0:0:0:0 with SMTP id i18csp1847710ile; Thu, 6 Feb 2020 09:47:52 -0800 (PST) X-Google-Smtp-Source: APXvYqyLCGOYlJMJCSCjEsRFmkZ6PnBE82MWLXULltL1u/wji5tKGgRAqiHWpVpgYlvGk+rILw8b X-Received: by 2002:a37:6706:: with SMTP id b6mr3425426qkc.461.1581011272000; Thu, 06 Feb 2020 09:47:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1581011271; cv=none; d=google.com; s=arc-20160816; b=zw/morUac4aFDzmAjFKRdUQQ0ap7EP7N7Ww1s6YBlpJromTvlU/1kFzTQB+ycUQ6gV GNKxoSXTySNCmvOE2nYU8iwlD8tWFuKoXe0jM8h81ymFiVmWzz8msftTcElhpN91/WIy Dsn3F3gDsTPcQyxx35lziW+/6Qlr2+8uxwbbFhWFuFgorOxBFYf60F+6mpdPQbbSUxI6 tETZcfUu7Uh8QE3U5bu53al6t91OW5jsv1Xxo3fu03ZcwT037doqh9unrhtnupdnpgjd vIg1RTu7i1WKNhRm4doJUW492zlYkyN1YAmazPFNuEPiPQz3dVluerHckBAyR34LR28e B56w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=uG5NDXEGg3myTzZttWMRSQ2riVD+Wnq5tJN9+Gr0JO8=; b=mY7f8TmjliZ9HZcA4zsSmPcfgdjR15W2j4Dr1r49Fftxb4BOkUhedawgR4vdQ9rcNj +unzY4jPNxROBc+3vHH9FUSRqrXwH5mkEuWUkkS5bKJJE0ivtXlA2YWfi9JyHiVyFnRq a/WAx/NwItWcOMFpqAX/HpHCHEgH/EofLi2Wh9sZTMhrMJKbfFYbCX2bCmMqw0pHGM00 FMsw/MkdzSbaIngeCYaAFT21UJw6HkA1sb6xHYg3wHNjLGsIl70U+f0o6EuneUl3erTy 8D2xxgCD8qk+ZYDs7MgPb+7o6yLVcVY2z03s1fJimg15FZ/cxGNHmFAhsHjo8M0CZ5zc 270Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=HPnRFdK9; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a68si1937186qke.236.2020.02.06.09.47.51 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 06 Feb 2020 09:47:51 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=HPnRFdK9; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:43736 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izlFn-0000vP-EO for patch@linaro.org; Thu, 06 Feb 2020 12:47:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:43652) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izl08-0006u1-Vq for qemu-devel@nongnu.org; Thu, 06 Feb 2020 12:31:44 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1izl05-0006oh-CO for qemu-devel@nongnu.org; Thu, 06 Feb 2020 12:31:40 -0500 Received: from mail-wm1-x32a.google.com ([2a00:1450:4864:20::32a]:51969) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1izl03-0006dz-KT for qemu-devel@nongnu.org; Thu, 06 Feb 2020 12:31:37 -0500 Received: by mail-wm1-x32a.google.com with SMTP id t23so902361wmi.1 for ; Thu, 06 Feb 2020 09:31:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uG5NDXEGg3myTzZttWMRSQ2riVD+Wnq5tJN9+Gr0JO8=; b=HPnRFdK9v97HwNjA4B3poqNfGABq1ap6SUXDpGg+vWCinCBZx3FB+bIgbSv/eES2z/ Uj7wq41REfVF9ffgmNoWTSM45J8C2BOSBwVkrqi0ncL9YTSTW2e3MYIESuqkfQCuS0g3 x39OITu1A2G5DXcU9hJuLXgJRloksM6TaVJp4VlHAYTVyh7G9gcERt1JW4LfrfM2QKOQ 5VsCmeFjBoBsuff62JBZyc7ZXklgFMA6ZmbUMUXFnK8Q/PAgrIan7ia+VUI3Qv87tLi/ qybnbf5Jfd3KpamawVzyAbYZEN4a01RqhjhQsQnL+eAZax6QnLMj5ROKM38dxxTnNHyi IKlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uG5NDXEGg3myTzZttWMRSQ2riVD+Wnq5tJN9+Gr0JO8=; b=ad12Pzm8hSXYTgO+USD1mKD+kwSvRJh1Rug8VGye/WCcg6m8MgwTj+dv4xex+wPObh E6Q3B85PKn92p7WiQAhFYcyDUdIcaHt2UlKIfGOuk4yCmx8MAhpzLTiAljNGk79xwarO vvFWaFEZWxWIA1NaEjhtmc4lLKalgn6EbRrsYnXRDknhqJHP2ZkDMNMCFvY1SyYQeGhv ONnmvsUkeYtidW4UzJOnejkdGezRw8kqxpgCKlnoNl2nLqQAyYAN3qoV9Tlf46C2RrDQ FapujLX3xpfH2MCXEnq2anPYPrtmDUxsNWdnmgf850FN+6VB1lTrs+AaV6qgfsOLm31t ZhaQ== X-Gm-Message-State: APjAAAUzhbkPwBnsnmoATt3iWyqWTNj9AQ1n3G+Bp0/sgafeYuToTXKB sKyZZyt0jsZoHnCg3AhrxWxiRQtmYz8= X-Received: by 2002:a1c:113:: with SMTP id 19mr5930397wmb.95.1581010293124; Thu, 06 Feb 2020 09:31:33 -0800 (PST) Received: from orth.archaic.org.uk (orth.archaic.org.uk. [81.2.115.148]) by smtp.gmail.com with ESMTPSA id w15sm5204214wrs.80.2020.02.06.09.31.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 06 Feb 2020 09:31:32 -0800 (PST) From: Peter Maydell To: qemu-devel@nongnu.org Subject: [PATCH 22/29] scripts/qapi/parser.py: improve doc comment indent handling Date: Thu, 6 Feb 2020 17:30:33 +0000 Message-Id: <20200206173040.17337-23-peter.maydell@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200206173040.17337-1-peter.maydell@linaro.org> References: <20200206173040.17337-1-peter.maydell@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::32a X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Markus Armbruster , Michael Roth , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Make the handling of indentation in doc comments more sophisticated, so that when we see a section like: Notes: some text some more text indented line 3 we save it for the doc-comment processing code as: some text some more text indented line 3 and when we see a section with the heading on its own line: Notes: some text some more text indented text we also accept that and save it in the same form. The exception is that we always retain indentation as-is for Examples sections, because these are literal text. If we detect that the comment document text is not indented as much as we expect it to be, we throw a parse error. (We don't complain about over-indented sections, because for rST this can be legitimate markup.) Signed-off-by: Peter Maydell --- scripts/qapi/parser.py | 82 +++++++++++++++++++++++++++++++++--------- 1 file changed, 65 insertions(+), 17 deletions(-) -- 2.20.1 diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 2196ec5de1e..66f802641c9 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -313,18 +313,32 @@ class QAPIDoc(object): """ class Section(object): - def __init__(self, name=None): + def __init__(self, parser, name=None, indent=0): + # parser, for error messages about indentation + self._parser = parser # optional section name (argument/member or section name) self.name = name # the list of lines for this section self.text = '' + # the expected indent level of the text of this section + self._indent = indent def append(self, line): + # Strip leading spaces corresponding to the expected indent level + # Blank lines are always OK. + if line: + spacecount = len(line) - len(line.lstrip(" ")) + if spacecount > self._indent: + spacecount = self._indent + if spacecount < self._indent: + raise QAPIParseError(self._parser, "unexpected de-indent") + line = line[spacecount:] + self.text += line.rstrip() + '\n' class ArgSection(Section): - def __init__(self, name): - QAPIDoc.Section.__init__(self, name) + def __init__(self, parser, name, indent=0): + QAPIDoc.Section.__init__(self, parser, name, indent) self.member = None def connect(self, member): @@ -338,7 +352,7 @@ class QAPIDoc(object): self._parser = parser self.info = info self.symbol = None - self.body = QAPIDoc.Section() + self.body = QAPIDoc.Section(parser) # dict mapping parameter name to ArgSection self.args = OrderedDict() self.features = OrderedDict() @@ -443,7 +457,18 @@ class QAPIDoc(object): if name.startswith('@') and name.endswith(':'): line = line[len(name)+1:] - self._start_args_section(name[1:-1]) + if not line or line.isspace(): + # Line was just the "@arg:" header; following lines + # are not indented + indent = 0 + line = '' + else: + # Line is "@arg: first line of description"; following + # lines should be indented by len(name) + 1, and we + # pad out this first line so it is handled the same way + indent = len(name) + 1 + line = ' ' * indent + line + self._start_args_section(name[1:-1], indent) elif self._is_section_tag(name): self._append_line = self._append_various_line self._append_various_line(line) @@ -465,7 +490,17 @@ class QAPIDoc(object): if name.startswith('@') and name.endswith(':'): line = line[len(name)+1:] - self._start_features_section(name[1:-1]) + if not line or line.isspace(): + # Line is just the "@name:" header, no ident for following lines + indent = 0 + line = '' + else: + # Line is "@arg: first line of description"; following + # lines should be indented by len(name) + 3, and we + # pad out this first line so it is handled the same way + indent = len(name) + 1 + line = ' ' * indent + line + self._start_features_section(name[1:-1], indent) elif self._is_section_tag(name): self._append_line = self._append_various_line self._append_various_line(line) @@ -498,11 +533,23 @@ class QAPIDoc(object): % (name, self.sections[0].name)) elif self._is_section_tag(name): line = line[len(name)+1:] - self._start_section(name[:-1]) + if not line or line.isspace(): + # Line is just "SectionName:", no indent for following lines + indent = 0 + line = '' + elif name.startswith("Example"): + # The "Examples" section is literal-text, so preserve + # all the indentation as-is + indent = 0 + else: + # Line is "SectionName: some text", indent required + indent = len(name) + 1 + line = ' ' * indent + line + self._start_section(name[:-1], indent) self._append_freeform(line) - def _start_symbol_section(self, symbols_dict, name): + def _start_symbol_section(self, symbols_dict, name, indent): # FIXME invalid names other than the empty string aren't flagged if not name: raise QAPIParseError(self._parser, "invalid parameter name") @@ -511,21 +558,21 @@ class QAPIDoc(object): "'%s' parameter name duplicated" % name) assert not self.sections self._end_section() - self._section = QAPIDoc.ArgSection(name) + self._section = QAPIDoc.ArgSection(self._parser, name, indent) symbols_dict[name] = self._section - def _start_args_section(self, name): - self._start_symbol_section(self.args, name) + def _start_args_section(self, name, indent): + self._start_symbol_section(self.args, name, indent) - def _start_features_section(self, name): - self._start_symbol_section(self.features, name) + def _start_features_section(self, name, indent): + self._start_symbol_section(self.features, name, indent) - def _start_section(self, name=None): + def _start_section(self, name=None, indent=0): if name in ('Returns', 'Since') and self.has_section(name): raise QAPIParseError(self._parser, "duplicated '%s' section" % name) self._end_section() - self._section = QAPIDoc.Section(name) + self._section = QAPIDoc.Section(self._parser, name, indent) self.sections.append(self._section) def _end_section(self): @@ -548,7 +595,7 @@ class QAPIDoc(object): def connect_member(self, member): if member.name not in self.args: # Undocumented TODO outlaw - self.args[member.name] = QAPIDoc.ArgSection(member.name) + self.args[member.name] = QAPIDoc.ArgSection(self._parser, member.name) self.args[member.name].connect(member) def connect_feature(self, feature): @@ -556,7 +603,8 @@ class QAPIDoc(object): raise QAPISemError(feature.info, "feature '%s' lacks documentation" % feature.name) - self.features[feature.name] = QAPIDoc.ArgSection(feature.name) + self.features[feature.name] = QAPIDoc.ArgSection(self._parser, + feature.name) self.features[feature.name].connect(feature) def check_expr(self, expr):