From patchwork Wed Feb 14 18:40:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Corbet X-Patchwork-Id: 128376 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp911606ljc; Wed, 14 Feb 2018 10:40:40 -0800 (PST) X-Google-Smtp-Source: AH8x225zHTgdAu8YFABXYc+gyhC6crQcyPan/xFfz51z1+X4vXXmJBynP7gQIRUd747TeJwghB3O X-Received: by 2002:a17:902:bf0a:: with SMTP id bi10-v6mr68320plb.181.1518633640344; Wed, 14 Feb 2018 10:40:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518633640; cv=none; d=google.com; s=arc-20160816; b=FphqbFO/1wWoah8kDbV6bqvZYAafFZN+53VQtMyK6tWYCukEVUumcPBrib4D/6hIQI 5jBhxi2KzwRRj1wwf5q8wSYSaHdqGXydhWpDiDG1Pslyc+B0dhllWrNbgVWgb9t/sjUp nmSAWysQimwnxRZ9hdVI4vTUFupX+6yh8W4AVG08JL5S4Dpm5oidwMZOszpAD4bhw/s9 qtogxNujmUaZ/i4HI6yIetW5s2JCxJaOvhe+n+/Ld0T2TrXOzOabUcH/x4p0iYHCG5ux c3BtJf4tYEoWC9opA5mxh8cLxKYozrKgjL0utipmIjYOkUY0Do+1iq8i46ceeRdK352Z IjnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=mzHdl7Kl0kYH6ZnVQYW4MRSAXUDVtc5YThl1WAlyrHM=; b=WuVwAb80wQZXsTiDWmBcu79d8EQUdUsMiWOYg/Bm6W4BX0/J+P8TFIfcKpJDIW9uj8 GrAOKX0MXCIA7xfIfBsBAlIJxOTXkpUhiWe8x4hL6OOzq16Jya5JrHPGsCUmkw00fgv1 q9OgLQfx4osA2qjRPwU64I4Hn/fz/aE5k/Lnqy43weWBqHoEvVVqvb+aCZorupd7XulU gTbHj81hiW6RAU0HGZEy/iLs6GESCaZ5DSap8pnqnmcFehGjviuMlD5X6o1UY0FZpYc4 2p++Eq04wgdxKUHQ2fsustYH5wUZLBOA8Pjn4J+WAdQWrq5r4PsGSGwbKz3RJMivNcoM kl5g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j2-v6si1558312plk.264.2018.02.14.10.40.39; Wed, 14 Feb 2018 10:40:40 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1162235AbeBNSke (ORCPT + 28 others); Wed, 14 Feb 2018 13:40:34 -0500 Received: from ms.lwn.net ([45.79.88.28]:54642 "EHLO ms.lwn.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1161773AbeBNSk3 (ORCPT ); Wed, 14 Feb 2018 13:40:29 -0500 Received: from tpad.lan (localhost [127.0.0.1]) by ms.lwn.net (Postfix) with ESMTPA id AD5C62E7; Wed, 14 Feb 2018 18:40:28 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, mchehab@kernel.org, me@tobin.cc, jani.nikula@linux.intel.com, Jonathan Corbet Subject: [PATCH 1/8] docs: kernel-doc: Get rid of xml_escape() and friends Date: Wed, 14 Feb 2018 11:40:02 -0700 Message-Id: <20180214184009.12657-2-corbet@lwn.net> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180214184009.12657-1-corbet@lwn.net> References: <20180214184009.12657-1-corbet@lwn.net> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org XML escaping is a worry that came with DocBook, which we no longer have any dealings with. So get rid of the useless xml_escape()/xml_unescape() functions. No change to the generated output. Signed-off-by: Jonathan Corbet --- scripts/kernel-doc | 65 ++++++++---------------------------------------------- 1 file changed, 9 insertions(+), 56 deletions(-) -- 2.14.3 Reviewed-by: Jani Nikula diff --git a/scripts/kernel-doc b/scripts/kernel-doc index fee8952037b1..5aa4ce211fc6 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -553,10 +553,9 @@ sub output_highlight { } if ($line eq ""){ if (! $output_preformatted) { - print $lineprefix, local_unescape($blankline); + print $lineprefix, $blankline; } } else { - $line =~ s/\\\\\\/\&/g; if ($output_mode eq "man" && substr($line, 0, 1) eq ".") { print "\\&$line"; } else { @@ -751,9 +750,6 @@ sub output_highlight_rst { my $contents = join "\n",@_; my $line; - # undo the evil effects of xml_escape() earlier - $contents = xml_unescape($contents); - eval $dohighlight; die $@ if $@; @@ -1422,8 +1418,6 @@ sub push_parameter($$$$) { } } - $param = xml_escape($param); - # strip spaces from $param so that it is one continuous string # on @parameterlist; # this fixes a problem where check_sections() cannot find @@ -1748,47 +1742,6 @@ sub process_proto_type($$) { } } -# xml_escape: replace <, >, and & in the text stream; -# -# however, formatting controls that are generated internally/locally in the -# kernel-doc script are not escaped here; instead, they begin life like -# $blankline_html (4 of '\' followed by a mnemonic + ':'), then these strings -# are converted to their mnemonic-expected output, without the 4 * '\' & ':', -# just before actual output; (this is done by local_unescape()) -sub xml_escape($) { - my $text = shift; - if ($output_mode eq "man") { - return $text; - } - $text =~ s/\&/\\\\\\amp;/g; - $text =~ s/\/\\\\\\gt;/g; - return $text; -} - -# xml_unescape: reverse the effects of xml_escape -sub xml_unescape($) { - my $text = shift; - if ($output_mode eq "man") { - return $text; - } - $text =~ s/\\\\\\amp;/\&/g; - $text =~ s/\\\\\\lt;//g; - return $text; -} - -# convert local escape strings to html -# local escape strings look like: '\\\\menmonic:' (that's 4 backslashes) -sub local_unescape($) { - my $text = shift; - if ($output_mode eq "man") { - return $text; - } - $text =~ s/\\\\\\\\lt://g; - return $text; -} sub map_filename($) { my $file; @@ -1889,7 +1842,7 @@ sub process_file($) { $descr =~ s/^\s*//; $descr =~ s/\s*$//; $descr =~ s/\s+/ /g; - $declaration_purpose = xml_escape($descr); + $declaration_purpose = $descr; $in_purpose = 1; } else { $declaration_purpose = ""; @@ -1944,7 +1897,7 @@ sub process_file($) { print STDERR "${file}:$.: warning: contents before sections\n"; ++$warnings; } - dump_section($file, $section, xml_escape($contents)); + dump_section($file, $section, $contents); $section = $section_default; } @@ -1962,7 +1915,7 @@ sub process_file($) { $leading_space = undef; } elsif (/$doc_end/) { if (($contents ne "") && ($contents ne "\n")) { - dump_section($file, $section, xml_escape($contents)); + dump_section($file, $section, $contents); $section = $section_default; $contents = ""; } @@ -1981,7 +1934,7 @@ sub process_file($) { # @parameter line to signify start of description if ($1 eq "") { if ($section =~ m/^@/ || $section eq $section_context) { - dump_section($file, $section, xml_escape($contents)); + dump_section($file, $section, $contents); $section = $section_default; $contents = ""; $new_start_line = $.; @@ -1992,7 +1945,7 @@ sub process_file($) { } elsif ($in_purpose == 1) { # Continued declaration purpose chomp($declaration_purpose); - $declaration_purpose .= " " . xml_escape($1); + $declaration_purpose .= " " . $1; $declaration_purpose =~ s/\s+/ /g; } else { my $cont = $1; @@ -2030,7 +1983,7 @@ sub process_file($) { # Documentation block end */ } elsif (/$doc_inline_end/) { if (($contents ne "") && ($contents ne "\n")) { - dump_section($file, $section, xml_escape($contents)); + dump_section($file, $section, $contents); $section = $section_default; $contents = ""; } @@ -2057,7 +2010,7 @@ sub process_file($) { $contents = $2; if ($contents ne "") { $contents .= "\n"; - dump_section($file, $section, xml_escape($contents)); + dump_section($file, $section, $contents); $section = $section_default; $contents = ""; } @@ -2072,7 +2025,7 @@ sub process_file($) { } elsif ($state == STATE_DOCBLOCK) { if (/$doc_end/) { - dump_doc_section($file, $section, xml_escape($contents)); + dump_doc_section($file, $section, $contents); $section = $section_default; $contents = ""; $function = ""; From patchwork Wed Feb 14 18:40:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Corbet X-Patchwork-Id: 128381 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp912721ljc; Wed, 14 Feb 2018 10:42:03 -0800 (PST) X-Google-Smtp-Source: AH8x227RvKonP/6YAfdJ6uB+EROnuDjUTqfozS6FyRxxnDjnX9AcqCRqUTJq3a3oQPqDh3KL3lht X-Received: by 10.98.12.1 with SMTP id u1mr24933pfi.192.1518633723581; Wed, 14 Feb 2018 10:42:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518633723; cv=none; d=google.com; s=arc-20160816; b=TN4HmB4OyxQnG7x3yr+E71qGaMCBIoNhyw8K1FIysPDs2WD6Ev4HpQt6ho+CjILKbt PYistlAoPwJILZG5sdk3H3w242gRxvLmTDdB0AZ93WDoKb0T1mhRuJy8huFJ93kl2+hO c2bamCFw0b/F9m3klTe+lY7GQcn22umkaZw4cC3j92aSEfYFVHPu3n2CYXYQEM5um9Kh lRnVFyEvuRwOKn03TrxGautuTfvnhjXboB8cFRVeJXOvH1ZnMDqR7WDh4rAHWLtzouu7 fRuRzeBlwDmqNCQUJWJUfxdWqO6LeBYMAtKGiZq5SSWThILH4xlcjuImfWUhlAfiif/I 8//w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=8l6tvX91MiZtlPQCAAZS+eP6nFXt0uJq33A/7NlUdzY=; b=lIxVKrqEN6yDOlqkvNWtBfyIobBTxoGBifGcvuLIS+XDAFXv5yxKSrnmLMXqUY/PF6 hi0scFpsoDF6S8eL/7qlyCAoRrIIDmxz9joPYE5ch10B/KojDn/7m+5XjpoYwZF/mKYu f/IaiL84YYyGRcGUONIEdzojOerbJpaUzS3wDYnBO5hfj9PlPSMi7/FaIGDx9YQqe3Qx GNCiaehDdzNsE1bhIDC4azBAap8Et+EHVzQxeMnI/1JE1xuniLzgWFqYev4xAYfkQm7Q ArpkGuRyocFhtuufa+J+hiEsVgvkgLAA6OiMAp/9DRBn45BpqJRUT+ca3b4/Y4o48jFf nwdQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b89-v6si1562182plb.809.2018.02.14.10.42.03; Wed, 14 Feb 2018 10:42:03 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1162220AbeBNSkc (ORCPT + 28 others); Wed, 14 Feb 2018 13:40:32 -0500 Received: from ms.lwn.net ([45.79.88.28]:54656 "EHLO ms.lwn.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1162189AbeBNSka (ORCPT ); Wed, 14 Feb 2018 13:40:30 -0500 Received: from tpad.lan (localhost [127.0.0.1]) by ms.lwn.net (Postfix) with ESMTPA id 9679930A; Wed, 14 Feb 2018 18:40:29 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, mchehab@kernel.org, me@tobin.cc, jani.nikula@linux.intel.com, Jonathan Corbet Subject: [PATCH 2/8] docs: kernel-doc: Rename and split STATE_FIELD Date: Wed, 14 Feb 2018 11:40:03 -0700 Message-Id: <20180214184009.12657-3-corbet@lwn.net> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180214184009.12657-1-corbet@lwn.net> References: <20180214184009.12657-1-corbet@lwn.net> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org STATE_FIELD describes a parser state that can handle any part of a kerneldoc comment body; rename it to STATE_BODY to reflect that. The $in_purpose variable was a hidden substate of STATE_FIELD; get rid of it and make a proper state (STATE_BODY_MAYBE) instead. This will make the subsequent process_file() splitup easier. Reviewed-by: Jani Nikula Signed-off-by: Jonathan Corbet --- scripts/kernel-doc | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) -- 2.14.3 diff --git a/scripts/kernel-doc b/scripts/kernel-doc index 5aa4ce211fc6..ad30c52f91ef 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -328,10 +328,11 @@ my $lineprefix=""; use constant { STATE_NORMAL => 0, # normal code STATE_NAME => 1, # looking for function name - STATE_FIELD => 2, # scanning field start - STATE_PROTO => 3, # scanning prototype - STATE_DOCBLOCK => 4, # documentation block - STATE_INLINE => 5, # gathering documentation outside main block + STATE_BODY_MAYBE => 2, # body - or maybe more description + STATE_BODY => 3, # the body of the comment + STATE_PROTO => 4, # scanning prototype + STATE_DOCBLOCK => 5, # documentation block + STATE_INLINE => 6, # gathering documentation outside main block }; my $state; my $in_doc_sect; @@ -1784,7 +1785,6 @@ sub process_file($) { my $identifier; my $func; my $descr; - my $in_purpose = 0; my $initial_section_counter = $section_counter; my ($orig_file) = @_; my $leading_space; @@ -1830,7 +1830,7 @@ sub process_file($) { $identifier = $1; } - $state = STATE_FIELD; + $state = STATE_BODY; # if there's no @param blocks need to set up default section # here $contents = ""; @@ -1843,7 +1843,7 @@ sub process_file($) { $descr =~ s/\s*$//; $descr =~ s/\s+/ /g; $declaration_purpose = $descr; - $in_purpose = 1; + $state = STATE_BODY_MAYBE; } else { $declaration_purpose = ""; } @@ -1875,7 +1875,7 @@ sub process_file($) { ++$warnings; $state = STATE_NORMAL; } - } elsif ($state == STATE_FIELD) { # look for head: lines, and include content + } elsif ($state == STATE_BODY || $state == STATE_BODY_MAYBE) { if (/$doc_sect/i) { # case insensitive for supported section names $newsection = $1; $newcontents = $2; @@ -1902,7 +1902,7 @@ sub process_file($) { } $in_doc_sect = 1; - $in_purpose = 0; + $state = STATE_BODY; $contents = $newcontents; $new_start_line = $.; while (substr($contents, 0, 1) eq " ") { @@ -1941,8 +1941,8 @@ sub process_file($) { } else { $contents .= "\n"; } - $in_purpose = 0; - } elsif ($in_purpose == 1) { + $state = STATE_BODY; + } elsif ($state == STATE_BODY_MAYBE) { # Continued declaration purpose chomp($declaration_purpose); $declaration_purpose .= " " . $1; From patchwork Wed Feb 14 18:40:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Corbet X-Patchwork-Id: 128383 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp913138ljc; Wed, 14 Feb 2018 10:42:35 -0800 (PST) X-Google-Smtp-Source: AH8x225mjaFgvzOh++ikNeKWDnbunOkR2ypWIFNfNaM2tAaz7p1t+KstSlAbYPxmvrqZ9LHFCsrx X-Received: by 10.99.167.2 with SMTP id d2mr123565pgf.408.1518633755568; Wed, 14 Feb 2018 10:42:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518633755; cv=none; d=google.com; s=arc-20160816; b=YBFD9uVwTU7oy245i1acH6CFqY9W2UMPExf9IbUtGvH6vr1k6KjhskHZJmWVfKWBlK rhvH2+RTBbOy8v3LLGfylkIS/fdNls2ZlsOOv7FLha7reRWDWurX2SEza48Je857g07v Ohe6/l/9UUlvhXcgijIrWbf5CQ9bQimBjn6v+ODU7hWzwxPpzFTzDVEW7B8UppcjoqQ2 OHeXRMKoY7pVLhzLCOOuavXJmdYc6JEcM7g7q2vFA7TpluVbTqnYvKzvQxFzrSEZvHhZ d8NzpgLVnfA48SC8qnZduahd1xRhNTROyXsHXmIKDWV8JXpWjNpeFofY2mC2h2ZTTuBc V8jQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=PfHt6JD/rfcpJdAzhR8MX1RY2UvG02LoW8Brsgd6AQM=; b=aboqoIOtARRUPKf2pWDt87TvECxo0MzY57vPNGwJ3SHlqTdygrhl/Gcgpnl5Xq/NX/ f9qW5/n2MClsiRWUS1wp5jRj1t1zS69nVgW7WVMLO9cOunu86fNjZKsUlQFvnMNcGgvL El10pX3MymKHL6nKEUVe4OgaRcvylVc/OnkWNujS2FLoU8P3z5324fbzUbiAoXYfxzbq r5b97rTnxRAm8DMrQvqkukOoR6AglBB9HCdC4ekqAQnjAB1J5twjjMcMu/ojRUhnbXru TMBkE6nDyaworS5VkYx67FDwoVxRYUJsiSV7W8uvSYMK6boGzgajcJkhVGofsgJQDMCc y7yg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 1-v6si3529445plw.164.2018.02.14.10.42.35; Wed, 14 Feb 2018 10:42:35 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1162314AbeBNSm2 (ORCPT + 28 others); Wed, 14 Feb 2018 13:42:28 -0500 Received: from ms.lwn.net ([45.79.88.28]:54666 "EHLO ms.lwn.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1162199AbeBNSkb (ORCPT ); Wed, 14 Feb 2018 13:40:31 -0500 Received: from tpad.lan (localhost [127.0.0.1]) by ms.lwn.net (Postfix) with ESMTPA id 7C9C031D; Wed, 14 Feb 2018 18:40:30 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, mchehab@kernel.org, me@tobin.cc, jani.nikula@linux.intel.com, Jonathan Corbet Subject: [PATCH 3/8] docs: kernel-doc: Move STATE_NORMAL processing into its own function Date: Wed, 14 Feb 2018 11:40:04 -0700 Message-Id: <20180214184009.12657-4-corbet@lwn.net> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180214184009.12657-1-corbet@lwn.net> References: <20180214184009.12657-1-corbet@lwn.net> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Begin the process of splitting up the nearly 500-line process_file() function by moving STATE_NORMAL processing to a separate function. Reviewed-by: Jani Nikula Signed-off-by: Jonathan Corbet --- scripts/kernel-doc | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) -- 2.14.3 diff --git a/scripts/kernel-doc b/scripts/kernel-doc index ad30c52f91ef..65150b7c8472 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -1780,6 +1780,21 @@ sub process_export_file($) { close(IN); } +# +# Parsers for the various processing states. +# +# STATE_NORMAL: looking for the /** to begin everything. +# +sub process_normal() { + if (/$doc_start/o) { + $state = STATE_NAME; # next line is always the function name + $in_doc_sect = 0; + $declaration_start_line = $. + 1; + } +} + + + sub process_file($) { my $file; my $identifier; @@ -1807,11 +1822,7 @@ sub process_file($) { # Replace tabs by spaces while ($_ =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {}; if ($state == STATE_NORMAL) { - if (/$doc_start/o) { - $state = STATE_NAME; # next line is always the function name - $in_doc_sect = 0; - $declaration_start_line = $. + 1; - } + process_normal(); } elsif ($state == STATE_NAME) {# this line is the function name (always) if (/$doc_block/o) { $state = STATE_DOCBLOCK; From patchwork Wed Feb 14 18:40:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Corbet X-Patchwork-Id: 128382 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp912759ljc; Wed, 14 Feb 2018 10:42:06 -0800 (PST) X-Google-Smtp-Source: AH8x225tn3KSCMC60KLVvAYViMHJT3pV1KFlOHkVMb4hiWnaM9SluCQfztjp+/axNVj2NHgwGcE7 X-Received: by 10.101.67.2 with SMTP id j2mr116274pgq.159.1518633725964; Wed, 14 Feb 2018 10:42:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518633725; cv=none; d=google.com; s=arc-20160816; b=ApebV5oTtl4sNfY/XvKfMyA2dgD8ik6SnX8ZblqLCMWlOd1hC06Uqy7U3DaRPhlyPz rq7zefZ6WQvCx60UiWivKnQ6zeX/pI3IjWeqd9wQXdmz4SSyylRjEq75As+tEIgCD8Kp 328f6pl5VRHOcpJXT2sTy8zKylr9U69HqokAM+5X8CYa+AOU0HVGI3JmZB1IDiKWqKUy aL10arqDKESMmRIcBDY2VjA50aCtfJsuZB5zJTUzy3W/dnYxxs3IpHHRKnlnpJyXkWDW M4O4zOuN9EuNa8j/e10XtvO/BkoHWW2PjV/XpuhYhimvDExIyPLaENwSEjxsQt2JxICz zeTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=C+5Pl7ULW4GfCVD97MrzclLRpcCUgMkpboSNGezpfZ8=; b=crS0YpM9S3g2dv7eiQ3EIuOwdQsaVniNPlqP2zff8xsJQpKw6PfIgMwpnUi6IOb1M1 m5dCPCMC9nTzNdi1nLmO5vLjFQzyPzPZ5KE4fBOLEhfp4L73TQnP+EdYbIVrOLlgjPY0 r0cNrC3gWIIeV0iYe7ILAY1fnU0RYGAT8VaQdhIQPHPoSPvU0o3Wkmk9dm5rRInyVyPY bKU21vUyDyGlaBDmza3lOM7oe7EtnUUUvkbpaKDHJJwKnm736BupoC/fG+O1qSD85JBT QY4QWTCXL+hvosIWHPK61v7OoTpd5kg/JQppofmvCG7YzT9WrYRyjiY1x+jGITH0yK7A q/dw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id z12si2900233pgp.788.2018.02.14.10.42.05; Wed, 14 Feb 2018 10:42:05 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1162310AbeBNSmC (ORCPT + 28 others); Wed, 14 Feb 2018 13:42:02 -0500 Received: from ms.lwn.net ([45.79.88.28]:54672 "EHLO ms.lwn.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1162215AbeBNSkc (ORCPT ); Wed, 14 Feb 2018 13:40:32 -0500 Received: from tpad.lan (localhost [127.0.0.1]) by ms.lwn.net (Postfix) with ESMTPA id 52894537; Wed, 14 Feb 2018 18:40:31 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, mchehab@kernel.org, me@tobin.cc, jani.nikula@linux.intel.com, Jonathan Corbet Subject: [PATCH 4/8] docs: kernel-doc: Move STATE_NAME processing into its own function Date: Wed, 14 Feb 2018 11:40:05 -0700 Message-Id: <20180214184009.12657-5-corbet@lwn.net> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180214184009.12657-1-corbet@lwn.net> References: <20180214184009.12657-1-corbet@lwn.net> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Move this code out of process_file() in the name of readability and maintainability. Reviewed-by: Jani Nikula Signed-off-by: Jonathan Corbet --- scripts/kernel-doc | 137 ++++++++++++++++++++++++++++------------------------- 1 file changed, 72 insertions(+), 65 deletions(-) -- 2.14.3 diff --git a/scripts/kernel-doc b/scripts/kernel-doc index 65150b7c8472..a27c7016f72d 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -1793,13 +1793,81 @@ sub process_normal() { } } +# +# STATE_NAME: Looking for the "name - description" line +# +sub process_name($$) { + my $file = shift; + my $identifier; + my $descr; + + if (/$doc_block/o) { + $state = STATE_DOCBLOCK; + $contents = ""; + $new_start_line = $. + 1; + + if ( $1 eq "" ) { + $section = $section_intro; + } else { + $section = $1; + } + } + elsif (/$doc_decl/o) { + $identifier = $1; + if (/\s*([\w\s]+?)\s*-/) { + $identifier = $1; + } + $state = STATE_BODY; + # if there's no @param blocks need to set up default section + # here + $contents = ""; + $section = $section_default; + $new_start_line = $. + 1; + if (/-(.*)/) { + # strip leading/trailing/multiple spaces + $descr= $1; + $descr =~ s/^\s*//; + $descr =~ s/\s*$//; + $descr =~ s/\s+/ /g; + $declaration_purpose = $descr; + $state = STATE_BODY_MAYBE; + } else { + $declaration_purpose = ""; + } + + if (($declaration_purpose eq "") && $verbose) { + print STDERR "${file}:$.: warning: missing initial short description on line:\n"; + print STDERR $_; + ++$warnings; + } + + if ($identifier =~ m/^struct/) { + $decl_type = 'struct'; + } elsif ($identifier =~ m/^union/) { + $decl_type = 'union'; + } elsif ($identifier =~ m/^enum/) { + $decl_type = 'enum'; + } elsif ($identifier =~ m/^typedef/) { + $decl_type = 'typedef'; + } else { + $decl_type = 'function'; + } + + if ($verbose) { + print STDERR "${file}:$.: info: Scanning doc for $identifier\n"; + } + } else { + print STDERR "${file}:$.: warning: Cannot understand $_ on line $.", + " - I thought it was a doc line\n"; + ++$warnings; + $state = STATE_NORMAL; + } +} sub process_file($) { my $file; - my $identifier; my $func; - my $descr; my $initial_section_counter = $section_counter; my ($orig_file) = @_; my $leading_space; @@ -1823,69 +1891,8 @@ sub process_file($) { while ($_ =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {}; if ($state == STATE_NORMAL) { process_normal(); - } elsif ($state == STATE_NAME) {# this line is the function name (always) - if (/$doc_block/o) { - $state = STATE_DOCBLOCK; - $contents = ""; - $new_start_line = $. + 1; - - if ( $1 eq "" ) { - $section = $section_intro; - } else { - $section = $1; - } - } - elsif (/$doc_decl/o) { - $identifier = $1; - if (/\s*([\w\s]+?)\s*-/) { - $identifier = $1; - } - - $state = STATE_BODY; - # if there's no @param blocks need to set up default section - # here - $contents = ""; - $section = $section_default; - $new_start_line = $. + 1; - if (/-(.*)/) { - # strip leading/trailing/multiple spaces - $descr= $1; - $descr =~ s/^\s*//; - $descr =~ s/\s*$//; - $descr =~ s/\s+/ /g; - $declaration_purpose = $descr; - $state = STATE_BODY_MAYBE; - } else { - $declaration_purpose = ""; - } - - if (($declaration_purpose eq "") && $verbose) { - print STDERR "${file}:$.: warning: missing initial short description on line:\n"; - print STDERR $_; - ++$warnings; - } - - if ($identifier =~ m/^struct/) { - $decl_type = 'struct'; - } elsif ($identifier =~ m/^union/) { - $decl_type = 'union'; - } elsif ($identifier =~ m/^enum/) { - $decl_type = 'enum'; - } elsif ($identifier =~ m/^typedef/) { - $decl_type = 'typedef'; - } else { - $decl_type = 'function'; - } - - if ($verbose) { - print STDERR "${file}:$.: info: Scanning doc for $identifier\n"; - } - } else { - print STDERR "${file}:$.: warning: Cannot understand $_ on line $.", - " - I thought it was a doc line\n"; - ++$warnings; - $state = STATE_NORMAL; - } + } elsif ($state == STATE_NAME) { + process_name($file, $_); } elsif ($state == STATE_BODY || $state == STATE_BODY_MAYBE) { if (/$doc_sect/i) { # case insensitive for supported section names $newsection = $1; From patchwork Wed Feb 14 18:40:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Corbet X-Patchwork-Id: 128380 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp912533ljc; Wed, 14 Feb 2018 10:41:49 -0800 (PST) X-Google-Smtp-Source: AH8x226vs8p9CiHtlw8DpbVwwrv+GksrvgZqgf8kjm060Wl9Oi24iyTbwekHe0zqg68Cw63lL2Ap X-Received: by 10.98.58.204 with SMTP id v73mr41765pfj.0.1518633709479; Wed, 14 Feb 2018 10:41:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518633709; cv=none; d=google.com; s=arc-20160816; b=EB7FjZuAEK99T02pDpVn6PH0cxuC9+hzL/90jS0nGOuwaaC6JLAuuvn6miqFtXLclQ zyXWjH9rRzHe//ySncvbTrDt+v3Y+veGZKHVw7zEREPrWYERoPv7IcHuTz5Fg5H5OTU/ B7wJ3sLJfQJ9d1VdSEYdYrVa7cl3d6Lm2bigFoLq+bDjt3JG9gOrmWCAmOKaYGkP7JDE bc6ljDcS04/OFdEqURDlVkOx8HnQtLsBTWneyKYSUrK21y/Ynrg/4vZxNQMapeiZN432 bCYF8RATJ2aQO8j5oa9nzk4mu9BmnyCM0fRHiZJeuzjZ0mUBwwnHsFaoyO1zghn3RuXv cUVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=XLftaTLdRFxGmS/6nnn5eHS7vReB07uZSt9+M0CP5yE=; b=nyFEhh9UacExzvoo3XNmAHyb4caMMGH+EDN9ko4zxtO+rdQ+HP8R9v2kMuHPaAgEV+ /j+lFubY1bA2v/tcru1v8ZtCLGNfNA7iN1GDAQItTHfKxb4PX0KHuD2kpp0YlPgJMIFs tZa5GLY7wJIEIVLOPv1bRuFIrgy1kzkDUeSyGXrko7/Tvnw6hoWnVrBApzeQwrCEM3hB MiO1S021qqjTgiOJZtA3s+IKBBmaBktGGe1HbLz78vCfalWvepDwdQYdL8h7piYcunFb snYz6yK8Z+HWTpz7B0nPYFeH6a0ZPCCjOSXRhsDBgDir78gaphwphI4l6iypEiXEVvLr aCeA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id n59-v6si6247542plb.705.2018.02.14.10.41.49; Wed, 14 Feb 2018 10:41:49 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754737AbeBNSlq (ORCPT + 28 others); Wed, 14 Feb 2018 13:41:46 -0500 Received: from ms.lwn.net ([45.79.88.28]:54686 "EHLO ms.lwn.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1162189AbeBNSkd (ORCPT ); Wed, 14 Feb 2018 13:40:33 -0500 Received: from tpad.lan (localhost [127.0.0.1]) by ms.lwn.net (Postfix) with ESMTPA id 38B8012FB; Wed, 14 Feb 2018 18:40:32 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, mchehab@kernel.org, me@tobin.cc, jani.nikula@linux.intel.com, Jonathan Corbet Subject: [PATCH 5/8] docs: kernel-doc: Move STATE_BODY processing to a separate function Date: Wed, 14 Feb 2018 11:40:06 -0700 Message-Id: <20180214184009.12657-6-corbet@lwn.net> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180214184009.12657-1-corbet@lwn.net> References: <20180214184009.12657-1-corbet@lwn.net> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Also group the pseudo-global $leading_space variable with its peers. Reviewed-by: Jani Nikula Signed-off-by: Jonathan Corbet --- scripts/kernel-doc | 193 ++++++++++++++++++++++++++++------------------------- 1 file changed, 101 insertions(+), 92 deletions(-) -- 2.14.3 diff --git a/scripts/kernel-doc b/scripts/kernel-doc index a27c7016f72d..a6a7bb46ea29 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -336,6 +336,7 @@ use constant { }; my $state; my $in_doc_sect; +my $leading_space; # Inline documentation state use constant { @@ -1865,12 +1866,110 @@ sub process_name($$) { } } + +# +# STATE_BODY and STATE_BODY_MAYBE: the bulk of a kerneldoc comment. +# +sub process_body($$) { + my $file = shift; + + if (/$doc_sect/i) { # case insensitive for supported section names + $newsection = $1; + $newcontents = $2; + + # map the supported section names to the canonical names + if ($newsection =~ m/^description$/i) { + $newsection = $section_default; + } elsif ($newsection =~ m/^context$/i) { + $newsection = $section_context; + } elsif ($newsection =~ m/^returns?$/i) { + $newsection = $section_return; + } elsif ($newsection =~ m/^\@return$/) { + # special: @return is a section, not a param description + $newsection = $section_return; + } + + if (($contents ne "") && ($contents ne "\n")) { + if (!$in_doc_sect && $verbose) { + print STDERR "${file}:$.: warning: contents before sections\n"; + ++$warnings; + } + dump_section($file, $section, $contents); + $section = $section_default; + } + + $in_doc_sect = 1; + $state = STATE_BODY; + $contents = $newcontents; + $new_start_line = $.; + while (substr($contents, 0, 1) eq " ") { + $contents = substr($contents, 1); + } + if ($contents ne "") { + $contents .= "\n"; + } + $section = $newsection; + $leading_space = undef; + } elsif (/$doc_end/) { + if (($contents ne "") && ($contents ne "\n")) { + dump_section($file, $section, $contents); + $section = $section_default; + $contents = ""; + } + # look for doc_com + + doc_end: + if ($_ =~ m'\s*\*\s*[a-zA-Z_0-9:\.]+\*/') { + print STDERR "${file}:$.: warning: suspicious ending line: $_"; + ++$warnings; + } + + $prototype = ""; + $state = STATE_PROTO; + $brcount = 0; + } elsif (/$doc_content/) { + # miguel-style comment kludge, look for blank lines after + # @parameter line to signify start of description + if ($1 eq "") { + if ($section =~ m/^@/ || $section eq $section_context) { + dump_section($file, $section, $contents); + $section = $section_default; + $contents = ""; + $new_start_line = $.; + } else { + $contents .= "\n"; + } + $state = STATE_BODY; + } elsif ($state == STATE_BODY_MAYBE) { + # Continued declaration purpose + chomp($declaration_purpose); + $declaration_purpose .= " " . $1; + $declaration_purpose =~ s/\s+/ /g; + } else { + my $cont = $1; + if ($section =~ m/^@/ || $section eq $section_context) { + if (!defined $leading_space) { + if ($cont =~ m/^(\s+)/) { + $leading_space = $1; + } else { + $leading_space = ""; + } + } + $cont =~ s/^$leading_space//; + } + $contents .= $cont . "\n"; + } + } else { + # i dont know - bad line? ignore. + print STDERR "${file}:$.: warning: bad line: $_"; + ++$warnings; + } +} + + sub process_file($) { my $file; my $func; my $initial_section_counter = $section_counter; my ($orig_file) = @_; - my $leading_space; $file = map_filename($orig_file); @@ -1894,97 +1993,7 @@ sub process_file($) { } elsif ($state == STATE_NAME) { process_name($file, $_); } elsif ($state == STATE_BODY || $state == STATE_BODY_MAYBE) { - if (/$doc_sect/i) { # case insensitive for supported section names - $newsection = $1; - $newcontents = $2; - - # map the supported section names to the canonical names - if ($newsection =~ m/^description$/i) { - $newsection = $section_default; - } elsif ($newsection =~ m/^context$/i) { - $newsection = $section_context; - } elsif ($newsection =~ m/^returns?$/i) { - $newsection = $section_return; - } elsif ($newsection =~ m/^\@return$/) { - # special: @return is a section, not a param description - $newsection = $section_return; - } - - if (($contents ne "") && ($contents ne "\n")) { - if (!$in_doc_sect && $verbose) { - print STDERR "${file}:$.: warning: contents before sections\n"; - ++$warnings; - } - dump_section($file, $section, $contents); - $section = $section_default; - } - - $in_doc_sect = 1; - $state = STATE_BODY; - $contents = $newcontents; - $new_start_line = $.; - while (substr($contents, 0, 1) eq " ") { - $contents = substr($contents, 1); - } - if ($contents ne "") { - $contents .= "\n"; - } - $section = $newsection; - $leading_space = undef; - } elsif (/$doc_end/) { - if (($contents ne "") && ($contents ne "\n")) { - dump_section($file, $section, $contents); - $section = $section_default; - $contents = ""; - } - # look for doc_com + + doc_end: - if ($_ =~ m'\s*\*\s*[a-zA-Z_0-9:\.]+\*/') { - print STDERR "${file}:$.: warning: suspicious ending line: $_"; - ++$warnings; - } - - $prototype = ""; - $state = STATE_PROTO; - $brcount = 0; -# print STDERR "end of doc comment, looking for prototype\n"; - } elsif (/$doc_content/) { - # miguel-style comment kludge, look for blank lines after - # @parameter line to signify start of description - if ($1 eq "") { - if ($section =~ m/^@/ || $section eq $section_context) { - dump_section($file, $section, $contents); - $section = $section_default; - $contents = ""; - $new_start_line = $.; - } else { - $contents .= "\n"; - } - $state = STATE_BODY; - } elsif ($state == STATE_BODY_MAYBE) { - # Continued declaration purpose - chomp($declaration_purpose); - $declaration_purpose .= " " . $1; - $declaration_purpose =~ s/\s+/ /g; - } else { - my $cont = $1; - if ($section =~ m/^@/ || $section eq $section_context) { - if (!defined $leading_space) { - if ($cont =~ m/^(\s+)/) { - $leading_space = $1; - } else { - $leading_space = ""; - } - } - - $cont =~ s/^$leading_space//; - } - $contents .= $cont . "\n"; - } - } else { - # i dont know - bad line? ignore. - print STDERR "${file}:$.: warning: bad line: $_"; - ++$warnings; - } + process_body($file, $_); } elsif ($state == STATE_INLINE) { # scanning for inline parameters # First line (state 1) needs to be a @parameter if ($inline_doc_state == STATE_INLINE_NAME && /$doc_inline_sect/o) { From patchwork Wed Feb 14 18:40:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Corbet X-Patchwork-Id: 128379 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp911992ljc; Wed, 14 Feb 2018 10:41:11 -0800 (PST) X-Google-Smtp-Source: AH8x2250oXTSj7d0kFH1+wFlbYP6Ot5KSz7/fSDRP8J68AtC70JHvolOulALV1FQOYJPXdp83dwm X-Received: by 2002:a17:902:5a4a:: with SMTP id f10-v6mr85113plm.308.1518633671244; Wed, 14 Feb 2018 10:41:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518633671; cv=none; d=google.com; s=arc-20160816; b=kEZSVS6bgWbJuHNLLW9fzF9kDMYT71f0fOCri2LpUE96f+B1ORf27pjqzI3RfyPZt9 Kz6zLlroPxZltmXgqqOlma/ob5DSEgZBmiNGjzpAv6inlfTKvHvS8XPgZyNxMr42PPc5 e4fWpHHZrIeW+/6tJl1ynN6gu0p0xaEIR9eec3l4V5rVs1jF3sWmDNDqkJKCv/aavDON uO8S/KavzG224Tvs6aFiA0ODTxBpongJgusdj470nQ5OF78kswfjYKnSV+Ogkn5QTuW3 oCCMoIMZh7gHg/ws6nqKjZjdq/gTNdrPeW3/rbIVAPq9nRCCYzLm+VViHEXAuZQPBnV1 85Yw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=eU+wz/xM3FbxY7swThcq2Bm7vL4EJ49/jHIQPlnTL/Q=; b=XAgP9EDKiV3718E8oNXrp9rwqN8NpdSWv8BpC3gi1L6jlkicdf4jiir6javlonlGNw SnsysItaljvnaGJ87xARw6az9cYZNzaVxv2dcfwCFtQAAXhzQOHRvDOuJ9AxH/6DEeqo 5cxyW8v3Bgkbw1j5db1TqTWk6qfuQQk/Sq7jbmjMjbg23Re5eC6PzJ3wJQvs2godGiFA j/Z6MjKgx6cB7vEGHExefKXQPd6Ns7eixh8AF3ioDZr2CDiD8mkuYLLrPSeqJRzjMPXV BdLIKSHWLaM8j3I6PcuuqLkcaQ8lv4Zhm38VjMyS9Dr63iy0yEydg9kFEgwDdaAws184 0hXw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b5-v6si377075pli.695.2018.02.14.10.41.10; Wed, 14 Feb 2018 10:41:11 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1162282AbeBNSlI (ORCPT + 28 others); Wed, 14 Feb 2018 13:41:08 -0500 Received: from ms.lwn.net ([45.79.88.28]:54692 "EHLO ms.lwn.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1162229AbeBNSke (ORCPT ); Wed, 14 Feb 2018 13:40:34 -0500 Received: from tpad.lan (localhost [127.0.0.1]) by ms.lwn.net (Postfix) with ESMTPA id 0BC342E7; Wed, 14 Feb 2018 18:40:32 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, mchehab@kernel.org, me@tobin.cc, jani.nikula@linux.intel.com, Jonathan Corbet Subject: [PATCH 6/8] docs: kernel-doc: Move STATE_PROTO processing into its own function Date: Wed, 14 Feb 2018 11:40:07 -0700 Message-Id: <20180214184009.12657-7-corbet@lwn.net> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180214184009.12657-1-corbet@lwn.net> References: <20180214184009.12657-1-corbet@lwn.net> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Move the top-level prototype-processing code out of process_file(). Reviewed-by: Jani Nikula Signed-off-by: Jonathan Corbet --- scripts/kernel-doc | 46 ++++++++++++++++++++++++++++------------------ 1 file changed, 28 insertions(+), 18 deletions(-) -- 2.14.3 diff --git a/scripts/kernel-doc b/scripts/kernel-doc index a6a7bb46ea29..2deddb876156 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -1965,6 +1965,32 @@ sub process_body($$) { } +# +# STATE_PROTO: reading a function/whatever prototype. +# +sub process_proto($$) { + my $file = shift; + + if (/$doc_inline_oneline/) { + $section = $1; + $contents = $2; + if ($contents ne "") { + $contents .= "\n"; + dump_section($file, $section, $contents); + $section = $section_default; + $contents = ""; + } + } elsif (/$doc_inline_start/) { + $state = STATE_INLINE; + $inline_doc_state = STATE_INLINE_NAME; + } elsif ($decl_type eq 'function') { + process_proto_function($_, $file); + } else { + process_proto_type($_, $file); + } +} + + sub process_file($) { my $file; my $func; @@ -2031,24 +2057,8 @@ sub process_file($) { ++$warnings; } } - } elsif ($state == STATE_PROTO) { # scanning for function '{' (end of prototype) - if (/$doc_inline_oneline/) { - $section = $1; - $contents = $2; - if ($contents ne "") { - $contents .= "\n"; - dump_section($file, $section, $contents); - $section = $section_default; - $contents = ""; - } - } elsif (/$doc_inline_start/) { - $state = STATE_INLINE; - $inline_doc_state = STATE_INLINE_NAME; - } elsif ($decl_type eq 'function') { - process_proto_function($_, $file); - } else { - process_proto_type($_, $file); - } + } elsif ($state == STATE_PROTO) { + process_proto($file, $_); } elsif ($state == STATE_DOCBLOCK) { if (/$doc_end/) { From patchwork Wed Feb 14 18:40:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Corbet X-Patchwork-Id: 128378 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp911750ljc; Wed, 14 Feb 2018 10:40:55 -0800 (PST) X-Google-Smtp-Source: AH8x225UQ788sE1f5W9u7ZSq/vIeVPE5AS/Fgx50u7HOOqP45cpvZwvucZkk+aJtrjMEZi9YfkFo X-Received: by 10.98.97.198 with SMTP id v189mr28996pfb.110.1518633655674; Wed, 14 Feb 2018 10:40:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518633655; cv=none; d=google.com; s=arc-20160816; b=mgQ+l87p2OR6DooQc7eVz8LzYPUaIyWMw1Qlj7esekmtkazLK2SBDIzOukbpO8La44 GFbabVrIrEQoU1305ZWwBi/w2OXS2isOSLM1yXnxOfv0DB7HfQ5MPk/y0m7c9TYgOFu8 kt+onM62gGSTFUCXHMap/Wu74GhBWjQxJMQAfAFr4s3+wXx2W0d4BlSqIlLXc6cBhMk5 VByYCdGKOBrlfg+X+WeAKjIYbrUh/vC5uyUhFFIuSQ8bmJcrTXCBmqCa6I3ILN/kNs// zbsoTYH4gLrXNSy4ACnHYeUiQOpFwqQekiezoryN+MD++I3tGyXafKNh7EUX8S56nila lb/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=Rtj3GXRMsqJ1e/8gKh7TIvLrlPRfDevEHsPtduD9sjc=; b=JuT9l+jlzJcAUUclvOklrcBHmtgMG8QG2sXLbSWZbT1Kuv9iNPFohdfTti0akchcR6 xiD4V/oUVYAUWMGOmZVAPnYm2cHlAjvs2SO5yifDXh9gfsod4eOTbkm4IfCWN82nRoB8 Dfni/1BfRRO9alUz9YCP6Vj+jd+Q1SAwN28VVlOQVfF+aqZGoRjmIERxj6wzl7xDUBoX JB3vtYVjhVORPMjE59UpXVW2Fsm3QxgA+XkhitL5A/IXb2/3cWZRAEtDFDqifG2gRmzr 1CjaZGcwrwyT8G8tov3Dh6816U1NQTmYyYWhePKsH3eWw3JsqgFVwZ18lKCpMLurSP5G iUsg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id q10si1676051pgd.335.2018.02.14.10.40.55; Wed, 14 Feb 2018 10:40:55 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1162253AbeBNSkh (ORCPT + 28 others); Wed, 14 Feb 2018 13:40:37 -0500 Received: from ms.lwn.net ([45.79.88.28]:54702 "EHLO ms.lwn.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1161773AbeBNSke (ORCPT ); Wed, 14 Feb 2018 13:40:34 -0500 Received: from tpad.lan (localhost [127.0.0.1]) by ms.lwn.net (Postfix) with ESMTPA id D53C631D; Wed, 14 Feb 2018 18:40:33 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, mchehab@kernel.org, me@tobin.cc, jani.nikula@linux.intel.com, Jonathan Corbet Subject: [PATCH 7/8] docs: kernel-doc: Finish moving STATE_* code out of process_file() Date: Wed, 14 Feb 2018 11:40:08 -0700 Message-Id: <20180214184009.12657-8-corbet@lwn.net> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180214184009.12657-1-corbet@lwn.net> References: <20180214184009.12657-1-corbet@lwn.net> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Move STATE_INLINE and STATE_DOCBLOCK code out of process_file(), which now actually fits on a single screen. Delete an unused variable and add a couple of comments while I'm at it. Reviewed-by: Jani Nikula Signed-off-by: Jonathan Corbet --- scripts/kernel-doc | 139 +++++++++++++++++++++++++++++------------------------ 1 file changed, 77 insertions(+), 62 deletions(-) -- 2.14.3 diff --git a/scripts/kernel-doc b/scripts/kernel-doc index 2deddb876156..fb8fbdb25036 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -1990,10 +1990,80 @@ sub process_proto($$) { } } +# +# STATE_DOCBLOCK: within a DOC: block. +# +sub process_docblock($$) { + my $file = shift; + + if (/$doc_end/) { + dump_doc_section($file, $section, $contents); + $section = $section_default; + $contents = ""; + $function = ""; + %parameterdescs = (); + %parametertypes = (); + @parameterlist = (); + %sections = (); + @sectionlist = (); + $prototype = ""; + $state = STATE_NORMAL; + } elsif (/$doc_content/) { + if ( $1 eq "" ) { + $contents .= $blankline; + } else { + $contents .= $1 . "\n"; + } + } +} + +# +# STATE_INLINE: docbook comments within a prototype. +# +sub process_inline($$) { + my $file = shift; + + # First line (state 1) needs to be a @parameter + if ($inline_doc_state == STATE_INLINE_NAME && /$doc_inline_sect/o) { + $section = $1; + $contents = $2; + $new_start_line = $.; + if ($contents ne "") { + while (substr($contents, 0, 1) eq " ") { + $contents = substr($contents, 1); + } + $contents .= "\n"; + } + $inline_doc_state = STATE_INLINE_TEXT; + # Documentation block end */ + } elsif (/$doc_inline_end/) { + if (($contents ne "") && ($contents ne "\n")) { + dump_section($file, $section, $contents); + $section = $section_default; + $contents = ""; + } + $state = STATE_PROTO; + $inline_doc_state = STATE_INLINE_NA; + # Regular text + } elsif (/$doc_content/) { + if ($inline_doc_state == STATE_INLINE_TEXT) { + $contents .= $1 . "\n"; + # nuke leading blank lines + if ($contents =~ /^\s*$/) { + $contents = ""; + } + } elsif ($inline_doc_state == STATE_INLINE_NAME) { + $inline_doc_state = STATE_INLINE_ERROR; + print STDERR "${file}:$.: warning: "; + print STDERR "Incorrect use of kernel-doc format: $_"; + ++$warnings; + } + } +} + sub process_file($) { my $file; - my $func; my $initial_section_counter = $section_counter; my ($orig_file) = @_; @@ -2014,6 +2084,8 @@ sub process_file($) { } # Replace tabs by spaces while ($_ =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {}; + + # Hand this line to the appropriate state handler if ($state == STATE_NORMAL) { process_normal(); } elsif ($state == STATE_NAME) { @@ -2021,72 +2093,15 @@ sub process_file($) { } elsif ($state == STATE_BODY || $state == STATE_BODY_MAYBE) { process_body($file, $_); } elsif ($state == STATE_INLINE) { # scanning for inline parameters - # First line (state 1) needs to be a @parameter - if ($inline_doc_state == STATE_INLINE_NAME && /$doc_inline_sect/o) { - $section = $1; - $contents = $2; - $new_start_line = $.; - if ($contents ne "") { - while (substr($contents, 0, 1) eq " ") { - $contents = substr($contents, 1); - } - $contents .= "\n"; - } - $inline_doc_state = STATE_INLINE_TEXT; - # Documentation block end */ - } elsif (/$doc_inline_end/) { - if (($contents ne "") && ($contents ne "\n")) { - dump_section($file, $section, $contents); - $section = $section_default; - $contents = ""; - } - $state = STATE_PROTO; - $inline_doc_state = STATE_INLINE_NA; - # Regular text - } elsif (/$doc_content/) { - if ($inline_doc_state == STATE_INLINE_TEXT) { - $contents .= $1 . "\n"; - # nuke leading blank lines - if ($contents =~ /^\s*$/) { - $contents = ""; - } - } elsif ($inline_doc_state == STATE_INLINE_NAME) { - $inline_doc_state = STATE_INLINE_ERROR; - print STDERR "${file}:$.: warning: "; - print STDERR "Incorrect use of kernel-doc format: $_"; - ++$warnings; - } - } + process_inline($file, $_); } elsif ($state == STATE_PROTO) { process_proto($file, $_); } elsif ($state == STATE_DOCBLOCK) { - if (/$doc_end/) - { - dump_doc_section($file, $section, $contents); - $section = $section_default; - $contents = ""; - $function = ""; - %parameterdescs = (); - %parametertypes = (); - @parameterlist = (); - %sections = (); - @sectionlist = (); - $prototype = ""; - $state = STATE_NORMAL; - } - elsif (/$doc_content/) - { - if ( $1 eq "" ) - { - $contents .= $blankline; - } - else - { - $contents .= $1 . "\n"; - } - } + process_docblock($file, $_); } } + + # Make sure we got something interesting. if ($initial_section_counter == $section_counter) { if ($output_mode ne "none") { print STDERR "${file}:1: warning: no structured comments found\n"; From patchwork Wed Feb 14 18:40:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Corbet X-Patchwork-Id: 128377 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp911653ljc; Wed, 14 Feb 2018 10:40:45 -0800 (PST) X-Google-Smtp-Source: AH8x227aB+2uvOSQsclvL0MxQyL5CJO+x6kNjUvKidgDeJNxoPeyIjTz49wby7La/Sh5Fi13mgOD X-Received: by 10.99.120.65 with SMTP id t62mr113726pgc.125.1518633645147; Wed, 14 Feb 2018 10:40:45 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518633645; cv=none; d=google.com; s=arc-20160816; b=xQRI8k5PqY/Vo5MjuXeMCKS4Avp8gu46CQMKqYx/4yOgZpVis3dKvo3OacQidXd8fZ pK3i6O/kFYYpa21gGu3mjXUf6F46TO0tNRO5Or5z+A2m1C3ivP61kVPPdxP9/IsgRRzH RWaf3gh2TeNQl/XY0iY8t7BN8d/Qthhb3tACy6c+BigsKCyRjkXJ7NT8Yr5c0HFtGkTY kPEVINYiOyEp3pWqiZwLpuvflNEcfIdR0j+ZH3m6VpvKz2FwBkL5UvcDlG8XlthPbOiR kqI2VjO8krwWpXGXGAc38DDziUG4f98w0Fg4NNoeib3mNk3kTLCZbuk551GIeplr5e4t o9xQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=EoV3QGd6TD3g1PifcVcBp9GmkDYoYtKqWQFXIUB9eqE=; b=kuh6gwpLdQs5PwBfRTdO33+zl6XYIxKmrld37D5sCzhyeet3PJXXNZnuKRF2Fszy9o jXdCDqj5bqxXbSz99VDUY0y3oH5j2iL2b1WfY5iKoM9RxQ0ObtnigEaEuc/nHQbutG8n oK5/Cf7dCxwtF9ZZbTamrV50Q872pqrorIWkq6XzBsA58qRE64J2H/52+KFJNQIC8l2N 7KoSTb+eAo+2SvlwBxQmQoPeidD1Slgu7yc6z1eW3AqeyeK0O1SsLXhCakDBpCKkS9FM n6XO/gFUk5taGJ5TofRKSWFgYoEwM8wB85uQCvMBVqq84dSf9Zaxx3ngUYAuxNDJSJzr 7jzw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id v30si1598337pgo.360.2018.02.14.10.40.44; Wed, 14 Feb 2018 10:40:45 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1162267AbeBNSkj (ORCPT + 28 others); Wed, 14 Feb 2018 13:40:39 -0500 Received: from ms.lwn.net ([45.79.88.28]:54716 "EHLO ms.lwn.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1162237AbeBNSkf (ORCPT ); Wed, 14 Feb 2018 13:40:35 -0500 Received: from tpad.lan (localhost [127.0.0.1]) by ms.lwn.net (Postfix) with ESMTPA id A9FCC30A; Wed, 14 Feb 2018 18:40:34 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, mchehab@kernel.org, me@tobin.cc, jani.nikula@linux.intel.com, Jonathan Corbet Subject: [PATCH 8/8] docs: kernel-doc: Don't mangle literal code blocks in comments Date: Wed, 14 Feb 2018 11:40:09 -0700 Message-Id: <20180214184009.12657-9-corbet@lwn.net> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180214184009.12657-1-corbet@lwn.net> References: <20180214184009.12657-1-corbet@lwn.net> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org It can be useful to put code snippets into kerneldoc comments; that can be done with the "::" operator at the end of a line like this:: if (desperate) run_in_circles(); The ".. code-block::" directive can also be used to this end. kernel-doc currently fails to understand these literal blocks and applies its normal markup to them, which is then treated as literal by sphinx. The result is unsightly markup instead of a useful code snippet. Apply a hack to the output code to recognize literal blocks and avoid performing any special markup on them. It's ugly, but that means it fits in well with the rest of the script. Signed-off-by: Jonathan Corbet --- scripts/kernel-doc | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 64 insertions(+), 5 deletions(-) -- 2.14.3 Reviewed-by: Jani Nikula diff --git a/scripts/kernel-doc b/scripts/kernel-doc index fb8fbdb25036..cbe864e72a2f 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -748,14 +748,73 @@ sub output_blockhead_rst(%) { } } -sub output_highlight_rst { - my $contents = join "\n",@_; - my $line; - +# +# Apply the RST highlights to a sub-block of text. +# +sub highlight_block($) { + # The dohighlight kludge requires the text be called $contents + my $contents = shift; eval $dohighlight; die $@ if $@; + return $contents; +} - foreach $line (split "\n", $contents) { +# +# Regexes used only here. +# +my $sphinx_literal = '^[^.].*::$'; +my $sphinx_cblock = '^\.\.\ +code-block::'; + +sub output_highlight_rst { + my $input = join "\n",@_; + my $output = ""; + my $line; + my $in_literal = 0; + my $litprefix; + my $block = ""; + + foreach $line (split "\n",$input) { + # + # If we're in a literal block, see if we should drop out + # of it. Otherwise pass the line straight through unmunged. + # + if ($in_literal) { + if (! ($line =~ /^\s*$/)) { + # + # If this is the first non-blank line in a literal + # block we need to figure out what the proper indent is. + # + if ($litprefix eq "") { + $line =~ /^(\s*)/; + $litprefix = '^' . $1; + $output .= $line . "\n"; + } elsif (! ($line =~ /$litprefix/)) { + $in_literal = 0; + } else { + $output .= $line . "\n"; + } + } else { + $output .= $line . "\n"; + } + } + # + # Not in a literal block (or just dropped out) + # + if (! $in_literal) { + $block .= $line . "\n"; + if (($line =~ /$sphinx_literal/) || ($line =~ /$sphinx_cblock/)) { + $in_literal = 1; + $litprefix = ""; + $output .= highlight_block($block); + $block = "" + } + } + } + + if ($block) { + $output .= highlight_block($block); + } + foreach $line (split "\n", $output) { print $lineprefix . $line . "\n"; } }