From patchwork Wed Feb 7 17:26:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Corbet X-Patchwork-Id: 127201 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp684382ljc; Wed, 7 Feb 2018 09:28:40 -0800 (PST) X-Google-Smtp-Source: AH8x224f6Ca93NZ0j/gmRFlnh7W7Pq7BCKMyuQyrXsLJylh2rVcy0xQAkLE37yCa/2qllIBpkcB6 X-Received: by 10.99.3.6 with SMTP id 6mr5578179pgd.334.1518024520606; Wed, 07 Feb 2018 09:28:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518024520; cv=none; d=google.com; s=arc-20160816; b=pzEAAFoIv8FfKbTX4Z6a+oX46jsUxliErbXiIZ1jEv1ts+WakhsT1HrzlNwHlz7zvE 7sUm5F9V4UnaVaua7ys91KmrqBhpj9scFpw7e8u1lh3WN/7DtxrDZU50cWLs336/dk77 IDE3tBaHaD8yFTA2sHNyFH9E5SvVvuUjANQWiU4fa6rxvthmo4lgwQ+PoaA/+FyjEDkN Yu/wCFDPl0hIQ4LQPNWL/fZj1REIDKmtF8KpLEG5VltPUDVnTwIrkDYFKOpi3TXYB90K nYGhxX65c6cmCrUU1QHuQkcslhK1N2Y5HWMDq/MdPy8bOPXCyxthv9vNR+zj3/aYU0mg QA9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=mzHdl7Kl0kYH6ZnVQYW4MRSAXUDVtc5YThl1WAlyrHM=; b=JHuMESqJKt9CuVSJAMMHO8BxRVXdVxdd6sbjVGozodPUWghbVPabKkqckNjxVqI0aT dfu+NGsmxJOueMI8u3WVZ3gSCZy3ZdiC82LZtsp448bW+S0aLQRET7pfeXRgdUrGBmgv gf9e272MqTHqTaBjlYX9NnPQ6HSFf7GEAShQcgA4HTGf9k1JwPN6fCLSy/0xwn8lc/mZ UV+XFGv7RidoqKHM5Ir76n312aZD7C5BKeahjh0QD1Xhx8eyF40THcfQYYOXsUAviFoh PwpLTGI10dz4yrOOCU8VXt85wFYgwOXj+22d+4A0BmvLKwIyVlVVtQDKr1vttuw6ujNF RShg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g6-v6si384plj.394.2018.02.07.09.28.40; Wed, 07 Feb 2018 09:28:40 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754966AbeBGR2j (ORCPT + 18 others); Wed, 7 Feb 2018 12:28:39 -0500 Received: from ms.lwn.net ([45.79.88.28]:41614 "EHLO ms.lwn.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754412AbeBGR1B (ORCPT ); Wed, 7 Feb 2018 12:27:01 -0500 Received: from tpad.lan (localhost [127.0.0.1]) by ms.lwn.net (Postfix) with ESMTPA id BE93C2B2; Wed, 7 Feb 2018 17:26:59 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, mchehab@kernel.org, me@tobin.cc, Jonathan Corbet Subject: [PATCH 1/8] docs: kernel-doc: Get rid of xml_escape() and friends Date: Wed, 7 Feb 2018 10:26:17 -0700 Message-Id: <20180207172624.24555-2-corbet@lwn.net> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180207172624.24555-1-corbet@lwn.net> References: <20180207172624.24555-1-corbet@lwn.net> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org XML escaping is a worry that came with DocBook, which we no longer have any dealings with. So get rid of the useless xml_escape()/xml_unescape() functions. No change to the generated output. Signed-off-by: Jonathan Corbet --- scripts/kernel-doc | 65 ++++++++---------------------------------------------- 1 file changed, 9 insertions(+), 56 deletions(-) -- 2.14.3 diff --git a/scripts/kernel-doc b/scripts/kernel-doc index fee8952037b1..5aa4ce211fc6 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -553,10 +553,9 @@ sub output_highlight { } if ($line eq ""){ if (! $output_preformatted) { - print $lineprefix, local_unescape($blankline); + print $lineprefix, $blankline; } } else { - $line =~ s/\\\\\\/\&/g; if ($output_mode eq "man" && substr($line, 0, 1) eq ".") { print "\\&$line"; } else { @@ -751,9 +750,6 @@ sub output_highlight_rst { my $contents = join "\n",@_; my $line; - # undo the evil effects of xml_escape() earlier - $contents = xml_unescape($contents); - eval $dohighlight; die $@ if $@; @@ -1422,8 +1418,6 @@ sub push_parameter($$$$) { } } - $param = xml_escape($param); - # strip spaces from $param so that it is one continuous string # on @parameterlist; # this fixes a problem where check_sections() cannot find @@ -1748,47 +1742,6 @@ sub process_proto_type($$) { } } -# xml_escape: replace <, >, and & in the text stream; -# -# however, formatting controls that are generated internally/locally in the -# kernel-doc script are not escaped here; instead, they begin life like -# $blankline_html (4 of '\' followed by a mnemonic + ':'), then these strings -# are converted to their mnemonic-expected output, without the 4 * '\' & ':', -# just before actual output; (this is done by local_unescape()) -sub xml_escape($) { - my $text = shift; - if ($output_mode eq "man") { - return $text; - } - $text =~ s/\&/\\\\\\amp;/g; - $text =~ s/\/\\\\\\gt;/g; - return $text; -} - -# xml_unescape: reverse the effects of xml_escape -sub xml_unescape($) { - my $text = shift; - if ($output_mode eq "man") { - return $text; - } - $text =~ s/\\\\\\amp;/\&/g; - $text =~ s/\\\\\\lt;//g; - return $text; -} - -# convert local escape strings to html -# local escape strings look like: '\\\\menmonic:' (that's 4 backslashes) -sub local_unescape($) { - my $text = shift; - if ($output_mode eq "man") { - return $text; - } - $text =~ s/\\\\\\\\lt://g; - return $text; -} sub map_filename($) { my $file; @@ -1889,7 +1842,7 @@ sub process_file($) { $descr =~ s/^\s*//; $descr =~ s/\s*$//; $descr =~ s/\s+/ /g; - $declaration_purpose = xml_escape($descr); + $declaration_purpose = $descr; $in_purpose = 1; } else { $declaration_purpose = ""; @@ -1944,7 +1897,7 @@ sub process_file($) { print STDERR "${file}:$.: warning: contents before sections\n"; ++$warnings; } - dump_section($file, $section, xml_escape($contents)); + dump_section($file, $section, $contents); $section = $section_default; } @@ -1962,7 +1915,7 @@ sub process_file($) { $leading_space = undef; } elsif (/$doc_end/) { if (($contents ne "") && ($contents ne "\n")) { - dump_section($file, $section, xml_escape($contents)); + dump_section($file, $section, $contents); $section = $section_default; $contents = ""; } @@ -1981,7 +1934,7 @@ sub process_file($) { # @parameter line to signify start of description if ($1 eq "") { if ($section =~ m/^@/ || $section eq $section_context) { - dump_section($file, $section, xml_escape($contents)); + dump_section($file, $section, $contents); $section = $section_default; $contents = ""; $new_start_line = $.; @@ -1992,7 +1945,7 @@ sub process_file($) { } elsif ($in_purpose == 1) { # Continued declaration purpose chomp($declaration_purpose); - $declaration_purpose .= " " . xml_escape($1); + $declaration_purpose .= " " . $1; $declaration_purpose =~ s/\s+/ /g; } else { my $cont = $1; @@ -2030,7 +1983,7 @@ sub process_file($) { # Documentation block end */ } elsif (/$doc_inline_end/) { if (($contents ne "") && ($contents ne "\n")) { - dump_section($file, $section, xml_escape($contents)); + dump_section($file, $section, $contents); $section = $section_default; $contents = ""; } @@ -2057,7 +2010,7 @@ sub process_file($) { $contents = $2; if ($contents ne "") { $contents .= "\n"; - dump_section($file, $section, xml_escape($contents)); + dump_section($file, $section, $contents); $section = $section_default; $contents = ""; } @@ -2072,7 +2025,7 @@ sub process_file($) { } elsif ($state == STATE_DOCBLOCK) { if (/$doc_end/) { - dump_doc_section($file, $section, xml_escape($contents)); + dump_doc_section($file, $section, $contents); $section = $section_default; $contents = ""; $function = ""; From patchwork Wed Feb 7 17:26:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Corbet X-Patchwork-Id: 127194 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp683110ljc; Wed, 7 Feb 2018 09:27:09 -0800 (PST) X-Google-Smtp-Source: AH8x225xaJ5CIqpuOffemZNIPjklEwhv83Pg3ASQ7tUZtOpLPcP4vnCvxXEvWOPfy3clWIgI3tHq X-Received: by 2002:a17:902:4523:: with SMTP id m32-v6mr6752807pld.449.1518024429525; Wed, 07 Feb 2018 09:27:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518024429; cv=none; d=google.com; s=arc-20160816; b=edYk4KCTfcBp3Le812+nMGCKlUvND9Uigq+WWQAbHJJgTEJeQ5hDRhIs8dtwiSGBdG fl9lsWVmZAHN/sjx6WqAdgDCySXNpxSk4BbLeZr+RG5N2auHRAxz0MVn8prD8Oae4T79 Bg2DSIKEzS2PSSWOk9jCAfKEB9siQijT9YanzWSFQsEChVJsVO3qrARqskQjkCuV3POY ooVh/Luraublyof7NhvVTPosz45mj1UP8VKy7TlmXmfX83qQ+HvpawLpRMNLoUTNC+VU PzfeFQC0AXRwJ78VKmp/iuEIIkUxSwKeNpd9itpOUQrzkJWFssYVkSW0k06WKvatarz1 5Pew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=16tr8AtcAQ3CZwbCUg3wz03v6qIRwahoYMqZP/6RFKM=; b=bDkpnOIYYB7DDz9suMW+BqdOLeSmujx4+RK6rWDol5sijF9wOhg111gYSyN0BQrKEA EsUJYEYotNn75IMF51KM3BzNNbRQCa4xK7oRPL9e/953S1DX4yG7nH22+qJTBS1I8mY8 IVVI8tjOnFN4CMzT74wKkG1iKUNBU11vyaZT1UZOtt2WAiJDpAt+YAXe8AjWRCcCgCBx a6ZdLbNbGVaCoDp/lIzzMHduRbLdRUnA3Ktf841ZijXFhGy31fz9V+3v7QTfBtte+Qbq Md0qMGimwZ5wlc1esjtb4cgdJQ1tFVg5LPm5lC1M/wUAKVZBg1ukm2NJcv2Nutq3mAJY Kzaw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id t4si1194173pgr.205.2018.02.07.09.27.09; Wed, 07 Feb 2018 09:27:09 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754865AbeBGR1G (ORCPT + 18 others); Wed, 7 Feb 2018 12:27:06 -0500 Received: from ms.lwn.net ([45.79.88.28]:41626 "EHLO ms.lwn.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754466AbeBGR1B (ORCPT ); Wed, 7 Feb 2018 12:27:01 -0500 Received: from tpad.lan (localhost [127.0.0.1]) by ms.lwn.net (Postfix) with ESMTPA id A13BD491; Wed, 7 Feb 2018 17:27:00 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, mchehab@kernel.org, me@tobin.cc, Jonathan Corbet Subject: [PATCH 2/8] docs: kernel-doc: Rename and split STATE_FIELD Date: Wed, 7 Feb 2018 10:26:18 -0700 Message-Id: <20180207172624.24555-3-corbet@lwn.net> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180207172624.24555-1-corbet@lwn.net> References: <20180207172624.24555-1-corbet@lwn.net> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org STATE_FIELD describes a parser state that can handle any part of a kerneldoc comment body; rename it to STATE_BODY to reflect that. The $in_purpose variable was a hidden substate of STATE_FIELD; get rid of it and make a proper state (STATE_BODY_MAYBE) instead. This will make the subsequent process_file() splitup easier. Signed-off-by: Jonathan Corbet --- scripts/kernel-doc | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) -- 2.14.3 Reviewed-by: Jani Nikula diff --git a/scripts/kernel-doc b/scripts/kernel-doc index 5aa4ce211fc6..ad30c52f91ef 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -328,10 +328,11 @@ my $lineprefix=""; use constant { STATE_NORMAL => 0, # normal code STATE_NAME => 1, # looking for function name - STATE_FIELD => 2, # scanning field start - STATE_PROTO => 3, # scanning prototype - STATE_DOCBLOCK => 4, # documentation block - STATE_INLINE => 5, # gathering documentation outside main block + STATE_BODY_MAYBE => 2, # body - or maybe more description + STATE_BODY => 3, # the body of the comment + STATE_PROTO => 4, # scanning prototype + STATE_DOCBLOCK => 5, # documentation block + STATE_INLINE => 6, # gathering documentation outside main block }; my $state; my $in_doc_sect; @@ -1784,7 +1785,6 @@ sub process_file($) { my $identifier; my $func; my $descr; - my $in_purpose = 0; my $initial_section_counter = $section_counter; my ($orig_file) = @_; my $leading_space; @@ -1830,7 +1830,7 @@ sub process_file($) { $identifier = $1; } - $state = STATE_FIELD; + $state = STATE_BODY; # if there's no @param blocks need to set up default section # here $contents = ""; @@ -1843,7 +1843,7 @@ sub process_file($) { $descr =~ s/\s*$//; $descr =~ s/\s+/ /g; $declaration_purpose = $descr; - $in_purpose = 1; + $state = STATE_BODY_MAYBE; } else { $declaration_purpose = ""; } @@ -1875,7 +1875,7 @@ sub process_file($) { ++$warnings; $state = STATE_NORMAL; } - } elsif ($state == STATE_FIELD) { # look for head: lines, and include content + } elsif ($state == STATE_BODY || $state == STATE_BODY_MAYBE) { if (/$doc_sect/i) { # case insensitive for supported section names $newsection = $1; $newcontents = $2; @@ -1902,7 +1902,7 @@ sub process_file($) { } $in_doc_sect = 1; - $in_purpose = 0; + $state = STATE_BODY; $contents = $newcontents; $new_start_line = $.; while (substr($contents, 0, 1) eq " ") { @@ -1941,8 +1941,8 @@ sub process_file($) { } else { $contents .= "\n"; } - $in_purpose = 0; - } elsif ($in_purpose == 1) { + $state = STATE_BODY; + } elsif ($state == STATE_BODY_MAYBE) { # Continued declaration purpose chomp($declaration_purpose); $declaration_purpose .= " " . $1; From patchwork Wed Feb 7 17:26:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Corbet X-Patchwork-Id: 127198 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp683871ljc; Wed, 7 Feb 2018 09:28:00 -0800 (PST) X-Google-Smtp-Source: AH8x226d6KoN1/AdlyAvS63RWhunT49Qe4K6yL8Z7Mddgzk5awkazkEzhE3DZh/CTaYVlRe/ohZ0 X-Received: by 10.98.138.70 with SMTP id y67mr4395923pfd.176.1518024480386; Wed, 07 Feb 2018 09:28:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518024480; cv=none; d=google.com; s=arc-20160816; b=PCuEFc9soHNb5Jl5CvAgSZpYVaOxpiFsLm16gygPfG8Nj5YDRBAZkD0rNjW9moYQ/q tqcHSiK4xKkTNoR7My0O7VelPcpYeN1y24wULIVcxRM/SnHvvCccYOgMPrTqLNhSqnxM +RE7k7xbFarnnwPyPpjLWk8ZVrTYZFU40P9f3xV2LpKCcNjRHuazfQHrodTLqBq+cNM4 LoGrTRb5evFy4XaWq3folABd7lWNipZdyPzx48Rq5jy3Vxg5eZs8kC/W8DKDm9pd2MDQ ZgFtMhQEL9ogOLjG1Jb0XFWuNafl/jM8TJviSIeWys/vmM2QT17bmbed1UDE06pD94PH nQkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=gDuOAyOlwUGUfqkVwU8/MPJjf1bRd/vPq+FhlI4/1AA=; b=pLBycjFktYgFToci76SP1SypjCAXEqpVjsEcNw095oKTGPnOVdl/uMx1mpivUSdk9u JhcWx/o54v4Vg+idebouVamqev4fZOd8zCra9vt6T1uGPxCfpfcvXRPubm8lQpDuR6qv fFMKBxevrXV6nDOClTwYHhxWWDyMSLuGT5/CRr0CzptvEbA6l70xQIfmBYkm5EVUaCgA v4rz+QjsBQOf4qDC12j2WaK4Y2RgmtB51I10vyCyCNX2dInUVgDKv504hY7dtd7gX0VS SuybhMEfS7CORLilWCIPV/jFjFZwF2I2X3uYEcQJqLlowbLJzN31LMNbGHcNn0y8VQPA VdPw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f10si1176226pgu.783.2018.02.07.09.28.00; Wed, 07 Feb 2018 09:28:00 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754848AbeBGR1F (ORCPT + 18 others); Wed, 7 Feb 2018 12:27:05 -0500 Received: from ms.lwn.net ([45.79.88.28]:41634 "EHLO ms.lwn.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754831AbeBGR1C (ORCPT ); Wed, 7 Feb 2018 12:27:02 -0500 Received: from tpad.lan (localhost [127.0.0.1]) by ms.lwn.net (Postfix) with ESMTPA id 678751329; Wed, 7 Feb 2018 17:27:01 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, mchehab@kernel.org, me@tobin.cc, Jonathan Corbet Subject: [PATCH 3/8] docs: kernel-doc: Move STATE_NORMAL processing into its own function Date: Wed, 7 Feb 2018 10:26:19 -0700 Message-Id: <20180207172624.24555-4-corbet@lwn.net> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180207172624.24555-1-corbet@lwn.net> References: <20180207172624.24555-1-corbet@lwn.net> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Begin the process of splitting up the nearly 500-line process_file() function by moving STATE_NORMAL processing to a separate function. Signed-off-by: Jonathan Corbet --- scripts/kernel-doc | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) -- 2.14.3 Reviewed-by: Jani Nikula diff --git a/scripts/kernel-doc b/scripts/kernel-doc index ad30c52f91ef..65150b7c8472 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -1780,6 +1780,21 @@ sub process_export_file($) { close(IN); } +# +# Parsers for the various processing states. +# +# STATE_NORMAL: looking for the /** to begin everything. +# +sub process_normal() { + if (/$doc_start/o) { + $state = STATE_NAME; # next line is always the function name + $in_doc_sect = 0; + $declaration_start_line = $. + 1; + } +} + + + sub process_file($) { my $file; my $identifier; @@ -1807,11 +1822,7 @@ sub process_file($) { # Replace tabs by spaces while ($_ =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {}; if ($state == STATE_NORMAL) { - if (/$doc_start/o) { - $state = STATE_NAME; # next line is always the function name - $in_doc_sect = 0; - $declaration_start_line = $. + 1; - } + process_normal(); } elsif ($state == STATE_NAME) {# this line is the function name (always) if (/$doc_block/o) { $state = STATE_DOCBLOCK; From patchwork Wed Feb 7 17:26:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Corbet X-Patchwork-Id: 127200 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp684204ljc; Wed, 7 Feb 2018 09:28:27 -0800 (PST) X-Google-Smtp-Source: AH8x226/6cYPrSJh/F+quPIMhOtcCEpEVh+pTWRq5tKAMIQzu49Oh0kDxc9wKJBgwjATUt37QS0B X-Received: by 2002:a17:902:67c5:: with SMTP id g5-v6mr6777173pln.106.1518024507768; Wed, 07 Feb 2018 09:28:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518024507; cv=none; d=google.com; s=arc-20160816; b=rpEEG7w8xijp3YX4gLq6b63KXtvcCXCytNQq3AW/z9K/cUH3U7PjshBePmAJt2RWnw BL9s0NYvdL2VvtUGgGpiLG4/1NI7onkyDvmSw7NMYA05oOCPRAqugLrf2+N2HpMDWNW0 R5ZBYJNG+T1J3PvGoo9BT5dtc+fJrEAIJtEwJzC5Cf1XJYQugXinB35SlUfMRy9kTA7z jHgaloRg89HsmXhuJuhBYP/Jq9Zv+ZC5pipX3ZHug5vkfWvo2p3pdgl8rNMZ5PYk9tN4 k52j8t+p78jDNZjZ4Elm8RJJKppnrVYdP/tVhM5XaqGXoV4Byv6RDTuBgYhmmSd6Kld5 L6bw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=L34GV5JNoZqB6XE7wzdYnkPES/GYGbZ+z1VlkAdUvtY=; b=TJ72opJncaO/ozVCLGeEftr9AS5+4EEQyuP/FADLAECR0CSf3rsdqZYF2zWTQfIjCG QGB8bqoxNxCMWB8c6Kk7nYnrRr6Nm6AgNdWjc07tv5nr4dHDxT5ykZEsbqlxkCFJcp22 HDEu8m3S/ExZjZa9Hww0qHAqnWMPxY8cbJ6bXAdkdigeX61m6cWUiv9AH/ltZ8XJ/Uo8 GuhzvMvDjKAJzxn21KfByyYA0sO49x3E6Q6i9q5HvnuH74GyHu/R6+T7CrabewqHdrwk RmCNhfISzjGWsO9KzAvl6b9wjpv9dYVQYJMLCNhvWuh3QXCSqqt34TciLZdNYsuLdWGV V5Aw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x8-v6si1366659plo.616.2018.02.07.09.28.27; Wed, 07 Feb 2018 09:28:27 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754946AbeBGR20 (ORCPT + 18 others); Wed, 7 Feb 2018 12:28:26 -0500 Received: from ms.lwn.net ([45.79.88.28]:41640 "EHLO ms.lwn.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754833AbeBGR1D (ORCPT ); Wed, 7 Feb 2018 12:27:03 -0500 Received: from tpad.lan (localhost [127.0.0.1]) by ms.lwn.net (Postfix) with ESMTPA id 30781141B; Wed, 7 Feb 2018 17:27:02 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, mchehab@kernel.org, me@tobin.cc, Jonathan Corbet Subject: [PATCH 4/8] docs: kernel-doc: Move STATE_NAME processing into its own function Date: Wed, 7 Feb 2018 10:26:20 -0700 Message-Id: <20180207172624.24555-5-corbet@lwn.net> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180207172624.24555-1-corbet@lwn.net> References: <20180207172624.24555-1-corbet@lwn.net> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Move this code out of process_file() in the name of readability and maintainability. Signed-off-by: Jonathan Corbet --- scripts/kernel-doc | 137 ++++++++++++++++++++++++++++------------------------- 1 file changed, 72 insertions(+), 65 deletions(-) -- 2.14.3 Reviewed-by: Jani Nikula diff --git a/scripts/kernel-doc b/scripts/kernel-doc index 65150b7c8472..a27c7016f72d 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -1793,13 +1793,81 @@ sub process_normal() { } } +# +# STATE_NAME: Looking for the "name - description" line +# +sub process_name($$) { + my $file = shift; + my $identifier; + my $descr; + + if (/$doc_block/o) { + $state = STATE_DOCBLOCK; + $contents = ""; + $new_start_line = $. + 1; + + if ( $1 eq "" ) { + $section = $section_intro; + } else { + $section = $1; + } + } + elsif (/$doc_decl/o) { + $identifier = $1; + if (/\s*([\w\s]+?)\s*-/) { + $identifier = $1; + } + $state = STATE_BODY; + # if there's no @param blocks need to set up default section + # here + $contents = ""; + $section = $section_default; + $new_start_line = $. + 1; + if (/-(.*)/) { + # strip leading/trailing/multiple spaces + $descr= $1; + $descr =~ s/^\s*//; + $descr =~ s/\s*$//; + $descr =~ s/\s+/ /g; + $declaration_purpose = $descr; + $state = STATE_BODY_MAYBE; + } else { + $declaration_purpose = ""; + } + + if (($declaration_purpose eq "") && $verbose) { + print STDERR "${file}:$.: warning: missing initial short description on line:\n"; + print STDERR $_; + ++$warnings; + } + + if ($identifier =~ m/^struct/) { + $decl_type = 'struct'; + } elsif ($identifier =~ m/^union/) { + $decl_type = 'union'; + } elsif ($identifier =~ m/^enum/) { + $decl_type = 'enum'; + } elsif ($identifier =~ m/^typedef/) { + $decl_type = 'typedef'; + } else { + $decl_type = 'function'; + } + + if ($verbose) { + print STDERR "${file}:$.: info: Scanning doc for $identifier\n"; + } + } else { + print STDERR "${file}:$.: warning: Cannot understand $_ on line $.", + " - I thought it was a doc line\n"; + ++$warnings; + $state = STATE_NORMAL; + } +} sub process_file($) { my $file; - my $identifier; my $func; - my $descr; my $initial_section_counter = $section_counter; my ($orig_file) = @_; my $leading_space; @@ -1823,69 +1891,8 @@ sub process_file($) { while ($_ =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {}; if ($state == STATE_NORMAL) { process_normal(); - } elsif ($state == STATE_NAME) {# this line is the function name (always) - if (/$doc_block/o) { - $state = STATE_DOCBLOCK; - $contents = ""; - $new_start_line = $. + 1; - - if ( $1 eq "" ) { - $section = $section_intro; - } else { - $section = $1; - } - } - elsif (/$doc_decl/o) { - $identifier = $1; - if (/\s*([\w\s]+?)\s*-/) { - $identifier = $1; - } - - $state = STATE_BODY; - # if there's no @param blocks need to set up default section - # here - $contents = ""; - $section = $section_default; - $new_start_line = $. + 1; - if (/-(.*)/) { - # strip leading/trailing/multiple spaces - $descr= $1; - $descr =~ s/^\s*//; - $descr =~ s/\s*$//; - $descr =~ s/\s+/ /g; - $declaration_purpose = $descr; - $state = STATE_BODY_MAYBE; - } else { - $declaration_purpose = ""; - } - - if (($declaration_purpose eq "") && $verbose) { - print STDERR "${file}:$.: warning: missing initial short description on line:\n"; - print STDERR $_; - ++$warnings; - } - - if ($identifier =~ m/^struct/) { - $decl_type = 'struct'; - } elsif ($identifier =~ m/^union/) { - $decl_type = 'union'; - } elsif ($identifier =~ m/^enum/) { - $decl_type = 'enum'; - } elsif ($identifier =~ m/^typedef/) { - $decl_type = 'typedef'; - } else { - $decl_type = 'function'; - } - - if ($verbose) { - print STDERR "${file}:$.: info: Scanning doc for $identifier\n"; - } - } else { - print STDERR "${file}:$.: warning: Cannot understand $_ on line $.", - " - I thought it was a doc line\n"; - ++$warnings; - $state = STATE_NORMAL; - } + } elsif ($state == STATE_NAME) { + process_name($file, $_); } elsif ($state == STATE_BODY || $state == STATE_BODY_MAYBE) { if (/$doc_sect/i) { # case insensitive for supported section names $newsection = $1; From patchwork Wed Feb 7 17:26:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Corbet X-Patchwork-Id: 127199 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp683937ljc; Wed, 7 Feb 2018 09:28:05 -0800 (PST) X-Google-Smtp-Source: AH8x225A+bAXIcXH3x6oAUOrGTGXBKN9gVGRCa0dPs166M5eT4YCHsdKT7lpFALuFKFNWW4LN6X7 X-Received: by 10.98.98.1 with SMTP id w1mr6723466pfb.9.1518024484880; Wed, 07 Feb 2018 09:28:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518024484; cv=none; d=google.com; s=arc-20160816; b=PqxbgSTqpiqOPzse6k+l+drtgb3e0mPL1a1zBNenCjKJAJwS8hWFjJQnxMHQ3orqoP DdPMEsWaa1BgTj0mh2++0TNcfkfDlEBVNOPuZquhoRbXDgWgNkbhCfkn00fHUKAB89qX OvO5fKpr6UC+f67O2GrTc8eHexU23OGGMgNaj48wcbxKFuo2AHP2rdAT3oXrpQk7xnZX 3aHRmZQKdL6x83QFusxUnxs8tj1y6fcOIt+8T11/6SeAAyHTDaSagRGgwoX4B0Oe4NHX siseNu4cJtZsSa1cEmSNuLxN/yqFBV28JmGKt26qyvwJIBsOEvy7bMWpkjQz7J1UR9l0 YfIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=DJAklSxwQMKVgAJWcYu4/hVGBg7LYRdS/FuZTtT586g=; b=dLdx5J4EFgtl3gekySATkK9atNGtuULFLJIte6H8SHeifJUAPxC1DKCbiBQbw95et+ +tdi9ngjximv5lTkJ+T23c8MSvRceu/IWAuPcK/cHX5PuGaUS8kMl7vqg0Q0SWAYGiPb pglgeXb3tQ29W9DotdXG9t7j7RzCFqlDru6ndq7ClGYyxbLa/elQJ4O+DIGoc2vl9ZDL Fo31PRjCpaON5pgl6b8ZnTpU1s3KUsNxXxzfAfRLfZPhbjOidT+KIDrPHmwuX71Luzic c8+2T1aKmqq8+RLZApmCvq0aC5jenyRXW21n1YbZjZtoEGENrjDkCC284UeFKWXayVvo nsnw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f10si1176226pgu.783.2018.02.07.09.28.04; Wed, 07 Feb 2018 09:28:04 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754927AbeBGR2C (ORCPT + 18 others); Wed, 7 Feb 2018 12:28:02 -0500 Received: from ms.lwn.net ([45.79.88.28]:41646 "EHLO ms.lwn.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754843AbeBGR1E (ORCPT ); Wed, 7 Feb 2018 12:27:04 -0500 Received: from tpad.lan (localhost [127.0.0.1]) by ms.lwn.net (Postfix) with ESMTPA id AA8A8491; Wed, 7 Feb 2018 17:27:03 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, mchehab@kernel.org, me@tobin.cc, Jonathan Corbet Subject: [PATCH 5/8] docs: kernel-doc: Move STATE_BODY processing to a separate function Date: Wed, 7 Feb 2018 10:26:21 -0700 Message-Id: <20180207172624.24555-6-corbet@lwn.net> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180207172624.24555-1-corbet@lwn.net> References: <20180207172624.24555-1-corbet@lwn.net> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Also group the pseudo-global $leading_space variable with its peers. Signed-off-by: Jonathan Corbet --- scripts/kernel-doc | 193 ++++++++++++++++++++++++++++------------------------- 1 file changed, 101 insertions(+), 92 deletions(-) -- 2.14.3 Reviewed-by: Jani Nikula diff --git a/scripts/kernel-doc b/scripts/kernel-doc index a27c7016f72d..a6a7bb46ea29 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -336,6 +336,7 @@ use constant { }; my $state; my $in_doc_sect; +my $leading_space; # Inline documentation state use constant { @@ -1865,12 +1866,110 @@ sub process_name($$) { } } + +# +# STATE_BODY and STATE_BODY_MAYBE: the bulk of a kerneldoc comment. +# +sub process_body($$) { + my $file = shift; + + if (/$doc_sect/i) { # case insensitive for supported section names + $newsection = $1; + $newcontents = $2; + + # map the supported section names to the canonical names + if ($newsection =~ m/^description$/i) { + $newsection = $section_default; + } elsif ($newsection =~ m/^context$/i) { + $newsection = $section_context; + } elsif ($newsection =~ m/^returns?$/i) { + $newsection = $section_return; + } elsif ($newsection =~ m/^\@return$/) { + # special: @return is a section, not a param description + $newsection = $section_return; + } + + if (($contents ne "") && ($contents ne "\n")) { + if (!$in_doc_sect && $verbose) { + print STDERR "${file}:$.: warning: contents before sections\n"; + ++$warnings; + } + dump_section($file, $section, $contents); + $section = $section_default; + } + + $in_doc_sect = 1; + $state = STATE_BODY; + $contents = $newcontents; + $new_start_line = $.; + while (substr($contents, 0, 1) eq " ") { + $contents = substr($contents, 1); + } + if ($contents ne "") { + $contents .= "\n"; + } + $section = $newsection; + $leading_space = undef; + } elsif (/$doc_end/) { + if (($contents ne "") && ($contents ne "\n")) { + dump_section($file, $section, $contents); + $section = $section_default; + $contents = ""; + } + # look for doc_com + + doc_end: + if ($_ =~ m'\s*\*\s*[a-zA-Z_0-9:\.]+\*/') { + print STDERR "${file}:$.: warning: suspicious ending line: $_"; + ++$warnings; + } + + $prototype = ""; + $state = STATE_PROTO; + $brcount = 0; + } elsif (/$doc_content/) { + # miguel-style comment kludge, look for blank lines after + # @parameter line to signify start of description + if ($1 eq "") { + if ($section =~ m/^@/ || $section eq $section_context) { + dump_section($file, $section, $contents); + $section = $section_default; + $contents = ""; + $new_start_line = $.; + } else { + $contents .= "\n"; + } + $state = STATE_BODY; + } elsif ($state == STATE_BODY_MAYBE) { + # Continued declaration purpose + chomp($declaration_purpose); + $declaration_purpose .= " " . $1; + $declaration_purpose =~ s/\s+/ /g; + } else { + my $cont = $1; + if ($section =~ m/^@/ || $section eq $section_context) { + if (!defined $leading_space) { + if ($cont =~ m/^(\s+)/) { + $leading_space = $1; + } else { + $leading_space = ""; + } + } + $cont =~ s/^$leading_space//; + } + $contents .= $cont . "\n"; + } + } else { + # i dont know - bad line? ignore. + print STDERR "${file}:$.: warning: bad line: $_"; + ++$warnings; + } +} + + sub process_file($) { my $file; my $func; my $initial_section_counter = $section_counter; my ($orig_file) = @_; - my $leading_space; $file = map_filename($orig_file); @@ -1894,97 +1993,7 @@ sub process_file($) { } elsif ($state == STATE_NAME) { process_name($file, $_); } elsif ($state == STATE_BODY || $state == STATE_BODY_MAYBE) { - if (/$doc_sect/i) { # case insensitive for supported section names - $newsection = $1; - $newcontents = $2; - - # map the supported section names to the canonical names - if ($newsection =~ m/^description$/i) { - $newsection = $section_default; - } elsif ($newsection =~ m/^context$/i) { - $newsection = $section_context; - } elsif ($newsection =~ m/^returns?$/i) { - $newsection = $section_return; - } elsif ($newsection =~ m/^\@return$/) { - # special: @return is a section, not a param description - $newsection = $section_return; - } - - if (($contents ne "") && ($contents ne "\n")) { - if (!$in_doc_sect && $verbose) { - print STDERR "${file}:$.: warning: contents before sections\n"; - ++$warnings; - } - dump_section($file, $section, $contents); - $section = $section_default; - } - - $in_doc_sect = 1; - $state = STATE_BODY; - $contents = $newcontents; - $new_start_line = $.; - while (substr($contents, 0, 1) eq " ") { - $contents = substr($contents, 1); - } - if ($contents ne "") { - $contents .= "\n"; - } - $section = $newsection; - $leading_space = undef; - } elsif (/$doc_end/) { - if (($contents ne "") && ($contents ne "\n")) { - dump_section($file, $section, $contents); - $section = $section_default; - $contents = ""; - } - # look for doc_com + + doc_end: - if ($_ =~ m'\s*\*\s*[a-zA-Z_0-9:\.]+\*/') { - print STDERR "${file}:$.: warning: suspicious ending line: $_"; - ++$warnings; - } - - $prototype = ""; - $state = STATE_PROTO; - $brcount = 0; -# print STDERR "end of doc comment, looking for prototype\n"; - } elsif (/$doc_content/) { - # miguel-style comment kludge, look for blank lines after - # @parameter line to signify start of description - if ($1 eq "") { - if ($section =~ m/^@/ || $section eq $section_context) { - dump_section($file, $section, $contents); - $section = $section_default; - $contents = ""; - $new_start_line = $.; - } else { - $contents .= "\n"; - } - $state = STATE_BODY; - } elsif ($state == STATE_BODY_MAYBE) { - # Continued declaration purpose - chomp($declaration_purpose); - $declaration_purpose .= " " . $1; - $declaration_purpose =~ s/\s+/ /g; - } else { - my $cont = $1; - if ($section =~ m/^@/ || $section eq $section_context) { - if (!defined $leading_space) { - if ($cont =~ m/^(\s+)/) { - $leading_space = $1; - } else { - $leading_space = ""; - } - } - - $cont =~ s/^$leading_space//; - } - $contents .= $cont . "\n"; - } - } else { - # i dont know - bad line? ignore. - print STDERR "${file}:$.: warning: bad line: $_"; - ++$warnings; - } + process_body($file, $_); } elsif ($state == STATE_INLINE) { # scanning for inline parameters # First line (state 1) needs to be a @parameter if ($inline_doc_state == STATE_INLINE_NAME && /$doc_inline_sect/o) { From patchwork Wed Feb 7 17:26:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Corbet X-Patchwork-Id: 127197 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp683672ljc; Wed, 7 Feb 2018 09:27:48 -0800 (PST) X-Google-Smtp-Source: AH8x226Yz+GpS8i6hsCAa3MFyl7szlIwKflLtG3sr1ISLJ7o6I3kEyiFp0HI5u76FlSlCHK9zMcc X-Received: by 10.98.90.65 with SMTP id o62mr6737027pfb.69.1518024467935; Wed, 07 Feb 2018 09:27:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518024467; cv=none; d=google.com; s=arc-20160816; b=aORKnM0yWrenyqnZRwQyEynC9fHM+ZgnmMyBJZy7AxkOnFJpEyuYUGA56W274045QF VtHRrKp02wTbJQVQK/m6ruQTz4LVP17N7lT1rkCrwxO/Qg7CvtlUdVJ2zwbULnnhZYWh 9hBARMsfNKNqiGMhy9DoteJ/ZoFLBGYcqsyfJtPTqNck5Y8Rk6QuCb5OupNrTBrTkLqE kD2lO1c+rkQUAdPulqZ7XbI1dSCFcOOAWVmZI81rMU7lujlV60w/g4jflGxlqlR3q+io uH/KZFO0t9zycmFiGW2vuhYh15OO74W1QqdLqMliCkFux9j2LTU8WoG5XjWzaRthiUqj 8ZfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=bK9xpn7v/moMhewCJzAM8ReNkq4K1+N1+pWK5YvM/dY=; b=Jxv7fxf1fbEO7UuvyL4VSp5dwt2AUw6eti/QMzdSewMksi3NOiS2AvhUPir49/AiW8 UywrMhPiwuSw8f4x+Zc55GSEenOAK/PTFHvfYO7i132c03SRDuIhL335Zo/Asmea1buP r+ly4ce8zNGIQqR+2QYwkgVmb0Nn0VEF16EySsVKQRpotXVHZLPJgOdhI2E83K3Pi48I C5xn/2R9wjxXpAVZA2VD7KjH0klg22J1Wos5PlWPKIfpB+T/81e6tE+XdyQP5PMsjP7M A1G311ZyzFOM255gd2rSOAETY8GfBgDm9NcRiAxFwLB+2sw6Evutsy4GpJST+tZcVFQX UULQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c19-v6si1370618plo.278.2018.02.07.09.27.47; Wed, 07 Feb 2018 09:27:47 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754905AbeBGR1p (ORCPT + 18 others); Wed, 7 Feb 2018 12:27:45 -0500 Received: from ms.lwn.net ([45.79.88.28]:41654 "EHLO ms.lwn.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754831AbeBGR1F (ORCPT ); Wed, 7 Feb 2018 12:27:05 -0500 Received: from tpad.lan (localhost [127.0.0.1]) by ms.lwn.net (Postfix) with ESMTPA id ADE1A1421; Wed, 7 Feb 2018 17:27:04 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, mchehab@kernel.org, me@tobin.cc, Jonathan Corbet Subject: [PATCH 6/8] docs: kernel-doc: Move STATE_PROTO processing into its own function Date: Wed, 7 Feb 2018 10:26:22 -0700 Message-Id: <20180207172624.24555-7-corbet@lwn.net> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180207172624.24555-1-corbet@lwn.net> References: <20180207172624.24555-1-corbet@lwn.net> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Move the top-level prototype-processing code out of process_file(). Signed-off-by: Jonathan Corbet --- scripts/kernel-doc | 46 ++++++++++++++++++++++++++++------------------ 1 file changed, 28 insertions(+), 18 deletions(-) -- 2.14.3 Reviewed-by: Jani Nikula diff --git a/scripts/kernel-doc b/scripts/kernel-doc index a6a7bb46ea29..2deddb876156 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -1965,6 +1965,32 @@ sub process_body($$) { } +# +# STATE_PROTO: reading a function/whatever prototype. +# +sub process_proto($$) { + my $file = shift; + + if (/$doc_inline_oneline/) { + $section = $1; + $contents = $2; + if ($contents ne "") { + $contents .= "\n"; + dump_section($file, $section, $contents); + $section = $section_default; + $contents = ""; + } + } elsif (/$doc_inline_start/) { + $state = STATE_INLINE; + $inline_doc_state = STATE_INLINE_NAME; + } elsif ($decl_type eq 'function') { + process_proto_function($_, $file); + } else { + process_proto_type($_, $file); + } +} + + sub process_file($) { my $file; my $func; @@ -2031,24 +2057,8 @@ sub process_file($) { ++$warnings; } } - } elsif ($state == STATE_PROTO) { # scanning for function '{' (end of prototype) - if (/$doc_inline_oneline/) { - $section = $1; - $contents = $2; - if ($contents ne "") { - $contents .= "\n"; - dump_section($file, $section, $contents); - $section = $section_default; - $contents = ""; - } - } elsif (/$doc_inline_start/) { - $state = STATE_INLINE; - $inline_doc_state = STATE_INLINE_NAME; - } elsif ($decl_type eq 'function') { - process_proto_function($_, $file); - } else { - process_proto_type($_, $file); - } + } elsif ($state == STATE_PROTO) { + process_proto($file, $_); } elsif ($state == STATE_DOCBLOCK) { if (/$doc_end/) { From patchwork Wed Feb 7 17:26:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Corbet X-Patchwork-Id: 127195 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp683149ljc; Wed, 7 Feb 2018 09:27:11 -0800 (PST) X-Google-Smtp-Source: AH8x225o9isBmJ1AZncWwVTnigktT8qWk/nFwdmc29wjpcfizH+kC6A0TjP6x8nB7JDKrF17YuMF X-Received: by 10.99.123.81 with SMTP id k17mr5243428pgn.400.1518024431054; Wed, 07 Feb 2018 09:27:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518024431; cv=none; d=google.com; s=arc-20160816; b=iprV4hlmbuw8SEf1smick6NPRxQMZstp2z3pFKzbvWNXHyBu3ogmcC5mq+t0I6cIex ebYT83vFPwcLcgca22HiFxEfG5XOBoXiCMNH3vUTklXniySDCnBV9APXBR5o5FN9PeHb hNtxMLdhzyzZW5q30fva2jodpnwy+oHFxUmkagj3MmxOkCl2gKuvs9tlo2yyz8YA6ETr CC83iyIt20p1009UPQWY9Mypyn4CXqfqZP9Lg5lsuXYwPNho39Lw1oSfx/6OWWh/J1FQ efdwR2NZRaLZUPUzQhrjLyqyojbcNwW6kbFZVDGhOWM7xB0bRZtfYDA52vlAU5Uc2OmK 5oNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=VGHvoRSIdnvQVhBcO3JFklAZMduAYa5CxHUaKC5epXU=; b=fpYLrrpmWJr8vcAaOhcUb7nw+BWF94bPqvNbdVCxVb0YbCU9CAnwFmMAvGMITbBV6A 3FreGf3C3bKqmPaJXboiajMbawJpI2Q418Dnm+jpxE1TkNik87lz8UQMqPaaSSLITbiX HUzJYHKyxDLDeKaRjgdzjqIPOMBz2wXqJ2bV15B9ylSTXHulivqUSXyFY8IlwR0gsgWd eytVRltIp/DPhB2q6kA3bhsSpAHIFjfBUBZHse0MbSqAdhUUqAJtooUQuzhlK518Z8ss s3B2NlROhVRhCB3MKyn+RUhD1uhISkVfdV4it3nbR6BcCK458+qslmIXXgtwZ26qKSDb Tuqg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id t4si1194173pgr.205.2018.02.07.09.27.10; Wed, 07 Feb 2018 09:27:11 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754881AbeBGR1J (ORCPT + 18 others); Wed, 7 Feb 2018 12:27:09 -0500 Received: from ms.lwn.net ([45.79.88.28]:41660 "EHLO ms.lwn.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754860AbeBGR1G (ORCPT ); Wed, 7 Feb 2018 12:27:06 -0500 Received: from tpad.lan (localhost [127.0.0.1]) by ms.lwn.net (Postfix) with ESMTPA id 7465F1329; Wed, 7 Feb 2018 17:27:05 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, mchehab@kernel.org, me@tobin.cc, Jonathan Corbet Subject: [PATCH 7/8] docs: kernel-doc: Finish moving STATE_* code out of process_file() Date: Wed, 7 Feb 2018 10:26:23 -0700 Message-Id: <20180207172624.24555-8-corbet@lwn.net> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180207172624.24555-1-corbet@lwn.net> References: <20180207172624.24555-1-corbet@lwn.net> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Move STATE_INLINE and STATE_DOCBLOCK code out of process_file(), which now actually fits on a single screen. Delete an unused variable and add a couple of comments while I'm at it. Signed-off-by: Jonathan Corbet --- scripts/kernel-doc | 145 ++++++++++++++++++++++++++++++----------------------- 1 file changed, 83 insertions(+), 62 deletions(-) -- 2.14.3 Reviewed-by: Jani Nikula diff --git a/scripts/kernel-doc b/scripts/kernel-doc index 2deddb876156..c6c9370a1e49 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -1990,10 +1990,86 @@ sub process_proto($$) { } } +# +# STATE_DOCBLOCK: within a DOC: block. +# +sub process_docblock($$) { + my $file = shift; + + if (/$doc_end/) + { + dump_doc_section($file, $section, $contents); + $section = $section_default; + $contents = ""; + $function = ""; + %parameterdescs = (); + %parametertypes = (); + @parameterlist = (); + %sections = (); + @sectionlist = (); + $prototype = ""; + $state = STATE_NORMAL; + } + elsif (/$doc_content/) + { + if ( $1 eq "" ) + { + $contents .= $blankline; + } + else + { + $contents .= $1 . "\n"; + } + } +} + +# +# STATE_INLINE: docbook comments within a prototype. +# +sub process_inline($$) { + my $file = shift; + + # First line (state 1) needs to be a @parameter + if ($inline_doc_state == STATE_INLINE_NAME && /$doc_inline_sect/o) { + $section = $1; + $contents = $2; + $new_start_line = $.; + if ($contents ne "") { + while (substr($contents, 0, 1) eq " ") { + $contents = substr($contents, 1); + } + $contents .= "\n"; + } + $inline_doc_state = STATE_INLINE_TEXT; + # Documentation block end */ + } elsif (/$doc_inline_end/) { + if (($contents ne "") && ($contents ne "\n")) { + dump_section($file, $section, $contents); + $section = $section_default; + $contents = ""; + } + $state = STATE_PROTO; + $inline_doc_state = STATE_INLINE_NA; + # Regular text + } elsif (/$doc_content/) { + if ($inline_doc_state == STATE_INLINE_TEXT) { + $contents .= $1 . "\n"; + # nuke leading blank lines + if ($contents =~ /^\s*$/) { + $contents = ""; + } + } elsif ($inline_doc_state == STATE_INLINE_NAME) { + $inline_doc_state = STATE_INLINE_ERROR; + print STDERR "${file}:$.: warning: "; + print STDERR "Incorrect use of kernel-doc format: $_"; + ++$warnings; + } + } +} + sub process_file($) { my $file; - my $func; my $initial_section_counter = $section_counter; my ($orig_file) = @_; @@ -2014,6 +2090,8 @@ sub process_file($) { } # Replace tabs by spaces while ($_ =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {}; + + # Hand this line to the appropriate state handler if ($state == STATE_NORMAL) { process_normal(); } elsif ($state == STATE_NAME) { @@ -2021,72 +2099,15 @@ sub process_file($) { } elsif ($state == STATE_BODY || $state == STATE_BODY_MAYBE) { process_body($file, $_); } elsif ($state == STATE_INLINE) { # scanning for inline parameters - # First line (state 1) needs to be a @parameter - if ($inline_doc_state == STATE_INLINE_NAME && /$doc_inline_sect/o) { - $section = $1; - $contents = $2; - $new_start_line = $.; - if ($contents ne "") { - while (substr($contents, 0, 1) eq " ") { - $contents = substr($contents, 1); - } - $contents .= "\n"; - } - $inline_doc_state = STATE_INLINE_TEXT; - # Documentation block end */ - } elsif (/$doc_inline_end/) { - if (($contents ne "") && ($contents ne "\n")) { - dump_section($file, $section, $contents); - $section = $section_default; - $contents = ""; - } - $state = STATE_PROTO; - $inline_doc_state = STATE_INLINE_NA; - # Regular text - } elsif (/$doc_content/) { - if ($inline_doc_state == STATE_INLINE_TEXT) { - $contents .= $1 . "\n"; - # nuke leading blank lines - if ($contents =~ /^\s*$/) { - $contents = ""; - } - } elsif ($inline_doc_state == STATE_INLINE_NAME) { - $inline_doc_state = STATE_INLINE_ERROR; - print STDERR "${file}:$.: warning: "; - print STDERR "Incorrect use of kernel-doc format: $_"; - ++$warnings; - } - } + process_inline($file, $_); } elsif ($state == STATE_PROTO) { process_proto($file, $_); } elsif ($state == STATE_DOCBLOCK) { - if (/$doc_end/) - { - dump_doc_section($file, $section, $contents); - $section = $section_default; - $contents = ""; - $function = ""; - %parameterdescs = (); - %parametertypes = (); - @parameterlist = (); - %sections = (); - @sectionlist = (); - $prototype = ""; - $state = STATE_NORMAL; - } - elsif (/$doc_content/) - { - if ( $1 eq "" ) - { - $contents .= $blankline; - } - else - { - $contents .= $1 . "\n"; - } - } + process_docblock($file, $_); } } + + # Make sure we got something interesting. if ($initial_section_counter == $section_counter) { if ($output_mode ne "none") { print STDERR "${file}:1: warning: no structured comments found\n"; From patchwork Wed Feb 7 17:26:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Corbet X-Patchwork-Id: 127196 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp683351ljc; Wed, 7 Feb 2018 09:27:25 -0800 (PST) X-Google-Smtp-Source: AH8x227dUbI4ebAy6Yj//Fg1xaff2BDcQE8bdq4gaqq/n8//QOCljntZJJ7+VSG8HxWvlvodTzn4 X-Received: by 2002:a17:902:4101:: with SMTP id e1-v6mr6756152pld.332.1518024445605; Wed, 07 Feb 2018 09:27:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518024445; cv=none; d=google.com; s=arc-20160816; b=B3234pUIGqIio6dv0nN5z9BHbHtxyh5H1L852mZ952s93BACp+kUdMY1ZoCa0Td4ff DHOqR7Yk0zPqs9GbETzNvQ279oTbRdLakKPtjYda6JHQVXK4EBfoqEqcHdpfhCpBxuPb WqUyIh9IDNvbFuCtiBmje9MCJzTi1fSkgjCaBrOF/IQpuwneD7ECtey2y9MyQHG4yi/T nGO8qcwCQoMax8Vpzp2tMKu/XoMnlPrcvG56KPhSlEP3nDvuUDZmJUe5W2o99EMod2Sy J+x0j5zYEumGL6UzOCmdJJDGKCCdS3jYG6V2qbs/aO8ilq0SxrEViP8xgAOTJgbPGxfv 26wA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=n31qdi7pjZ397x6Y5RtOYzskiNK7wDwoEZX8imXtToI=; b=L6u0QCXatggybwNjSNfB9tBuf1a8NXqYARWxyhGbXXrKQcKXKy6OH3V5/pbdjDwpm0 UxiK13zUYhgim+c0FmBixq79MzWP/D1jNrIt0/0+gcJl8jYQvpxvz5VchxEyVZHq4f85 p1KWYdXDWI8XsOtWC6eLweJvbQUp8DFpz+SYd4rfOSEv35JCiIdTXcj/KYlDWEX2YV6p BMqknDJvRKQh0VGYL10VTp/N6NmmILf7GQuKIi9Un+qmei2Ys3q+R6I/lWwAyzLjwVPc WWLRpRTUPkMFwHIVxoAnPi8pmEPiHQsw4RvLeT8pfLm/AiVIX4bK7stNmEZpeOUYiQeK DOug== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c19-v6si1370618plo.278.2018.02.07.09.27.25; Wed, 07 Feb 2018 09:27:25 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754893AbeBGR1X (ORCPT + 18 others); Wed, 7 Feb 2018 12:27:23 -0500 Received: from ms.lwn.net ([45.79.88.28]:41668 "EHLO ms.lwn.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754466AbeBGR1H (ORCPT ); Wed, 7 Feb 2018 12:27:07 -0500 Received: from tpad.lan (localhost [127.0.0.1]) by ms.lwn.net (Postfix) with ESMTPA id 4766D2B2; Wed, 7 Feb 2018 17:27:06 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, mchehab@kernel.org, me@tobin.cc, Jonathan Corbet Subject: [PATCH 8/8] docs: kernel-doc: Don't mangle literal code blocks in comments Date: Wed, 7 Feb 2018 10:26:24 -0700 Message-Id: <20180207172624.24555-9-corbet@lwn.net> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180207172624.24555-1-corbet@lwn.net> References: <20180207172624.24555-1-corbet@lwn.net> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org It can be useful to put code snippets into kerneldoc comments; that can be done with the "::" operator at the end of a line like this:: if (desperate) run_in_circles(); kernel-doc currently fails to understand these literal blocks and applies its normal markup to them, which is then treated as literal by sphinx. The result is unsightly markup instead of a useful code snippet. Apply a hack to the output code to recognize literal blocks and avoid performing any special markup on them. It's ugly, but that means it fits in well with the rest of the script. Signed-off-by: Jonathan Corbet --- scripts/kernel-doc | 55 +++++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 50 insertions(+), 5 deletions(-) -- 2.14.3 diff --git a/scripts/kernel-doc b/scripts/kernel-doc index c6c9370a1e49..c984f82cb897 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -748,14 +748,59 @@ sub output_blockhead_rst(%) { } } -sub output_highlight_rst { - my $contents = join "\n",@_; - my $line; - +# +# Apply the RST highlights to a sub-block of text. +# +sub highlight_block($) { + # The dohighlight kludge requires the text be called $contents + my $contents = shift; eval $dohighlight; die $@ if $@; + return $contents; +} - foreach $line (split "\n", $contents) { +sub output_highlight_rst { + my $input = join "\n",@_; + my $output = ""; + my $line; + my $in_literal = 0; + my $litprefix; + my $block = ""; + + # The "dohighlight" hack requires that the data be called "$contents" + foreach $line (split "\n",$input) { + # + # If we're in a literal block, see if we should drop out + # of it. Otherwise pass the line straight through unmunged. + # + if ($in_literal) { + if (! ($line =~ /$litprefix/ || $line =~ /^\s*$/)) { + $in_literal = 0; + } + else { + $output .= $line . "\n"; + } + } + # + # Not in a literal block (or just dropped out) + # + if (! $in_literal) { + $block .= $line . "\n"; + if ($line =~ /^[^.].*::$/) { + $in_literal = 1; + # Note current indentation - we'll go as long as it's deeper. + $line =~ /^(\s*)/; + $litprefix = '^' . $1 . ' '; + $output .= highlight_block($block); + $block = "" + } + } + } + + if ($block) { + $output .= highlight_block($block); + } + foreach $line (split "\n", $output) { print $lineprefix . $line . "\n"; } }