| Message ID | 20190521211714.1395-1-corbet@lwn.net |
|---|---|
| Headers | show |
| Series | docs: Deal with some Sphinx deprecation warnings | expand |
On Tue, 21 May 2019, Jonathan Corbet <corbet@lwn.net> wrote: > The Sphinx folks are deprecating some interfaces in the upcoming 2.0 > release; one immediate result of that is a bunch of warnings that show up > when building with 1.8. These two patches make those warnings go away, > but at a cost: > > - It introduces a couple of Sphinx version checks, which are always > ugly, but the alternative would be to stop supporting versions > before 1.7. For now, I think we can carry that cruft. Frankly, I'd just require Sphinx 1.7+, available even in Debian stable through stretch-backports. > - The second patch causes the build to fail horribly on newer > Sphinx installations. The change to switch_source_input() seems > to make the parser much more finicky, increasing warnings and > eventually failing the build altogether. In particular, it will > scream about problems in .rst files that are not included in the > TOC tree at all. The complaints appear to be legitimate, but it's > a bunch of stuff to clean up. I can understand Sphinx complaining that a file is not included in a TOC tree, but I don't understand why it goes on to parse them anyway. BR, Jani. > > I've tested these with 1.4 and 1.8, but not various versions in between. > > Jonathan Corbet (2): > doc: Cope with Sphinx logging deprecations > doc: Cope with the deprecation of AutoReporter > > Documentation/sphinx/kerneldoc.py | 48 ++++++++++++++++++++++++------- > Documentation/sphinx/kernellog.py | 28 ++++++++++++++++++ > Documentation/sphinx/kfigure.py | 38 +++++++++++++----------- > 3 files changed, 87 insertions(+), 27 deletions(-) > create mode 100644 Documentation/sphinx/kernellog.py -- Jani Nikula, Intel Open Source Graphics Center
On Tue, May 21, 2019 at 03:17:12PM -0600, Jonathan Corbet wrote: > The Sphinx folks are deprecating some interfaces in the upcoming 2.0 > release; one immediate result of that is a bunch of warnings that show up > when building with 1.8. These two patches make those warnings go away, > but at a cost: A minor correction, if I may and if I understand this correctly: 2.0 is not an upcoming release, but a current one (2.0.1, to be precise), and this means that in some distros (like, Arch [1]) `make htmldocs` is already broken for quite some time. [1] https://bugs.archlinux.org/task/59688 > > - It introduces a couple of Sphinx version checks, which are always > ugly, but the alternative would be to stop supporting versions > before 1.7. For now, I think we can carry that cruft. > > - The second patch causes the build to fail horribly on newer > Sphinx installations. The change to switch_source_input() seems > to make the parser much more finicky, increasing warnings and > eventually failing the build altogether. In particular, it will > scream about problems in .rst files that are not included in the > TOC tree at all. The complaints appear to be legitimate, but it's > a bunch of stuff to clean up. > > I've tested these with 1.4 and 1.8, but not various versions in between. > > Jonathan Corbet (2): > doc: Cope with Sphinx logging deprecations > doc: Cope with the deprecation of AutoReporter > > Documentation/sphinx/kerneldoc.py | 48 ++++++++++++++++++++++++------- > Documentation/sphinx/kernellog.py | 28 ++++++++++++++++++ > Documentation/sphinx/kfigure.py | 38 +++++++++++++----------- > 3 files changed, 87 insertions(+), 27 deletions(-) > create mode 100644 Documentation/sphinx/kernellog.py > > -- > 2.21.0 > -- Best regards, Oleksandr Natalenko (post-factum) Senior Software Maintenance Engineer
On Wed, May 22, 2019 at 11:43:54AM +0200, Oleksandr Natalenko wrote: > On Tue, May 21, 2019 at 03:17:12PM -0600, Jonathan Corbet wrote: > > The Sphinx folks are deprecating some interfaces in the upcoming 2.0 > > release; one immediate result of that is a bunch of warnings that show up > > when building with 1.8. These two patches make those warnings go away, > > but at a cost: > > A minor correction, if I may and if I understand this correctly: 2.0 is > not an upcoming release, but a current one (2.0.1, to be precise), and > this means that in some distros (like, Arch [1]) `make htmldocs` is > already broken for quite some time. > > [1] https://bugs.archlinux.org/task/59688 ^^ this was the initial Bug for introducing the doc, but it got reverted in [2]. [2] https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/linux&id=cfe52e9aa8168d9571bedf8a376e2cfbd25223fd > > > > > - It introduces a couple of Sphinx version checks, which are always > > ugly, but the alternative would be to stop supporting versions > > before 1.7. For now, I think we can carry that cruft. > > > > - The second patch causes the build to fail horribly on newer > > Sphinx installations. The change to switch_source_input() seems > > to make the parser much more finicky, increasing warnings and > > eventually failing the build altogether. In particular, it will > > scream about problems in .rst files that are not included in the > > TOC tree at all. The complaints appear to be legitimate, but it's > > a bunch of stuff to clean up. > > > > I've tested these with 1.4 and 1.8, but not various versions in between. > > > > Jonathan Corbet (2): > > doc: Cope with Sphinx logging deprecations > > doc: Cope with the deprecation of AutoReporter > > > > Documentation/sphinx/kerneldoc.py | 48 ++++++++++++++++++++++++------- > > Documentation/sphinx/kernellog.py | 28 ++++++++++++++++++ > > Documentation/sphinx/kfigure.py | 38 +++++++++++++----------- > > 3 files changed, 87 insertions(+), 27 deletions(-) > > create mode 100644 Documentation/sphinx/kernellog.py > > > > -- > > 2.21.0 > > > > -- > Best regards, > Oleksandr Natalenko (post-factum) > Senior Software Maintenance Engineer -- Best regards, Oleksandr Natalenko (post-factum) Senior Software Maintenance Engineer
Em Wed, 22 May 2019 10:36:45 +0300 Jani Nikula <jani.nikula@linux.intel.com> escreveu: > On Tue, 21 May 2019, Jonathan Corbet <corbet@lwn.net> wrote: > > The Sphinx folks are deprecating some interfaces in the upcoming 2.0 > > release; one immediate result of that is a bunch of warnings that show up > > when building with 1.8. These two patches make those warnings go away, > > but at a cost: > > > > - It introduces a couple of Sphinx version checks, which are always > > ugly, but the alternative would be to stop supporting versions > > before 1.7. For now, I think we can carry that cruft. > > Frankly, I'd just require Sphinx 1.7+, available even in Debian stable > through stretch-backports. We can raise the bar and force a 1.7 or even 1.8 (Fedora 30 comes with version 1.8.4), but I would prefer to keep support older versions, at least while we don't depend on some new features introduced after the version we're using, and while our extensions won't require a major rework due to a new version. > > > - The second patch causes the build to fail horribly on newer > > Sphinx installations. The change to switch_source_input() seems > > to make the parser much more finicky, increasing warnings and > > eventually failing the build altogether. In particular, it will > > scream about problems in .rst files that are not included in the > > TOC tree at all. The complaints appear to be legitimate, but it's > > a bunch of stuff to clean up. There is a flag to cleanup the warning about a file not included at a TOC tree (:orphan:), but it will still try to parse it. There's also a conf.py way of doing it. For example, you can exclude an entire dir: exclude_patterns = ['includes/*.rst'] But using exclude_patterns will likely be too messy. > I can understand Sphinx complaining that a file is not included in a TOC > tree, but I don't understand why it goes on to parse them anyway. Yeah, this is, IMHO, a very bad behavior. > > BR, > Jani. > > > > > > I've tested these with 1.4 and 1.8, but not various versions in between. > > > > Jonathan Corbet (2): > > doc: Cope with Sphinx logging deprecations > > doc: Cope with the deprecation of AutoReporter > > > > Documentation/sphinx/kerneldoc.py | 48 ++++++++++++++++++++++++------- > > Documentation/sphinx/kernellog.py | 28 ++++++++++++++++++ > > Documentation/sphinx/kfigure.py | 38 +++++++++++++----------- > > 3 files changed, 87 insertions(+), 27 deletions(-) > > create mode 100644 Documentation/sphinx/kernellog.py > Thanks, Mauro
22.05.19 um 12:19 schrieb Mauro Carvalho Chehab: > Em Wed, 22 May 2019 10:36:45 +0300 > Jani Nikula <jani.nikula@linux.intel.com> escreveu: > >> On Tue, 21 May 2019, Jonathan Corbet <corbet@lwn.net> wrote: >>> The Sphinx folks are deprecating some interfaces in the upcoming 2.0 >>> release; one immediate result of that is a bunch of warnings that show up >>> when building with 1.8. These two patches make those warnings go away, >>> but at a cost: >>> >>> - It introduces a couple of Sphinx version checks, which are always >>> ugly, but the alternative would be to stop supporting versions >>> before 1.7. For now, I think we can carry that cruft. >> >> Frankly, I'd just require Sphinx 1.7+, available even in Debian stable >> through stretch-backports. > > We can raise the bar and force a 1.7 or even 1.8 (Fedora 30 comes > with version 1.8.4), but I would prefer to keep support older versions, > at least while we don't depend on some new features introduced after > the version we're using, and while our extensions won't require a major > rework due to a new version. Lets use 1.7 : - no need for Use_SSI wrapper - new log should work with 1.7 [1] --> no need for kernellog.py and additional imports, instead include on top of python modules :: from sphinx.util import logging logger = logging.getLogger('kerneldoc') [1] https://github.com/sphinx-doc/sphinx/commit/6d4e6454093953943e79d4db6efeb17390870e62 BTW we can drop other (old) sphinx-version issues e.g. Documentation/conf.py which fails with version 2.0: diff --git a/Documentation/conf.py b/Documentation/conf.py index 72647a38b5c2..ba82715b6715 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -34,13 +34,7 @@ needs_sphinx = '1.3' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. -extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure', 'sphinx.ext.ifconfig'] - -# The name of the math extension changed on Sphinx 1.4 -if major == 1 and minor > 3: - extensions.append("sphinx.ext.imgmath") -else: - extensions.append("sphinx.ext.pngmath") +extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure', 'sphinx.ext.ifconfig', 'sphinx.ext.imgmath'] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] -- Markus --
On Wed, 22 May 2019 15:25:36 +0200 Markus Heiser <markus.heiser@darmarit.de> wrote: > Lets use 1.7 : > > - no need for Use_SSI wrapper > - new log should work with 1.7 [1] --> no need for kernellog.py and > additional imports, instead include on top of python modules :: > > from sphinx.util import logging > logger = logging.getLogger('kerneldoc') I think we're going to have to drag things forward at some point in the not-too-distant future, but I think I'd rather not do that quite yet. The cost of supporting older sphinx for a few releases while we warn people is not all that high. So I think we should: - Put in (a future version of) my hacks for now, plus whatever else might be needed to make 2.0 work right. - Fix the fallout with regard to out-of-toctree .rst files so that we can actually build again with current sphinx. - Update Documentation/sphinx/requirements.txt to ask for something a wee bit more recent than 1.4.9. - Add a warning when building with an older version that (say) 1.7 will be required as of (say) 5.5. Does this make sense? Thanks, jon
Em Wed, 22 May 2019 09:45:59 -0600 Jonathan Corbet <corbet@lwn.net> escreveu: > On Wed, 22 May 2019 15:25:36 +0200 > Markus Heiser <markus.heiser@darmarit.de> wrote: > > > Lets use 1.7 : > > > > - no need for Use_SSI wrapper > > - new log should work with 1.7 [1] --> no need for kernellog.py and > > additional imports, instead include on top of python modules :: > > > > from sphinx.util import logging > > logger = logging.getLogger('kerneldoc') > > I think we're going to have to drag things forward at some point in the > not-too-distant future, but I think I'd rather not do that quite yet. The > cost of supporting older sphinx for a few releases while we warn people is > not all that high. So I think we should: > > - Put in (a future version of) my hacks for now, plus whatever else might > be needed to make 2.0 work right. > > - Fix the fallout with regard to out-of-toctree .rst files so that we can > actually build again with current sphinx. > > - Update Documentation/sphinx/requirements.txt to ask for something a wee > bit more recent than 1.4.9. You should remember to also update conf.py (with currently points to 1.3): # If your documentation needs a minimal Sphinx version, state it here. needs_sphinx = '1.3' Also, if you touch there, you should also touch: ./scripts/sphinx-pre-install The change there won't be as trivial as just changing this line: $virtenv_dir = "sphinx_1.4"; as the script should now run sphinx-build --version, in order to check if the version is lower than the new minimal version. It probably makes sense to make it grep the version from needs_sphinx at conf.py. > - Add a warning when building with an older version that (say) 1.7 will > be required as of (say) 5.5. It probably makes sense to add such check at the pre-install script, and add a: SPHINXOPTS="-jauto" somewhere if version is 1.7 or upper. > > Does this make sense? It makes sense to me. Thanks, Mauro
Em Wed, 22 May 2019 13:04:08 -0300 Mauro Carvalho Chehab <mchehab@kernel.org> escreveu: > Em Wed, 22 May 2019 09:45:59 -0600 > Jonathan Corbet <corbet@lwn.net> escreveu: > > > On Wed, 22 May 2019 15:25:36 +0200 > > Markus Heiser <markus.heiser@darmarit.de> wrote: > > > > > Lets use 1.7 : > > > > > > - no need for Use_SSI wrapper > > > - new log should work with 1.7 [1] --> no need for kernellog.py and > > > additional imports, instead include on top of python modules :: > > > > > > from sphinx.util import logging > > > logger = logging.getLogger('kerneldoc') > > > > I think we're going to have to drag things forward at some point in the > > not-too-distant future, but I think I'd rather not do that quite yet. The > > cost of supporting older sphinx for a few releases while we warn people is > > not all that high. So I think we should: > > > > - Put in (a future version of) my hacks for now, plus whatever else might > > be needed to make 2.0 work right. > > > > - Fix the fallout with regard to out-of-toctree .rst files so that we can > > actually build again with current sphinx. > > > > - Update Documentation/sphinx/requirements.txt to ask for something a wee > > bit more recent than 1.4.9. > > You should remember to also update conf.py (with currently points to 1.3): > > # If your documentation needs a minimal Sphinx version, state it here. > needs_sphinx = '1.3' > > Also, if you touch there, you should also touch: > > ./scripts/sphinx-pre-install > > The change there won't be as trivial as just changing this line: > > $virtenv_dir = "sphinx_1.4"; > > as the script should now run sphinx-build --version, in order to check > if the version is lower than the new minimal version. It probably makes > sense to make it grep the version from needs_sphinx at conf.py. > > > - Add a warning when building with an older version that (say) 1.7 will > > be required as of (say) 5.5. > > It probably makes sense to add such check at the pre-install script, > and add a: > > SPHINXOPTS="-jauto" > > somewhere if version is 1.7 or upper. > I'm meaning something like the enclosed patch. (PS.: I'm still working at the patch) Thanks, Mauro [RFC PATCH] scripts/sphinx-pre-install: make it handle Sphinx versions As we want to switch to a newer Sphinx version in the future, add some version detected logic. Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org> diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install index f6a5c0bae31e..8835aede4c61 100755 --- a/scripts/sphinx-pre-install +++ b/scripts/sphinx-pre-install @@ -13,7 +13,7 @@ use strict; # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. -my $virtenv_dir = "sphinx_1.4"; +my $conf = "Documentation/conf.py"; my $requirement_file = "Documentation/sphinx/requirements.txt"; # @@ -27,6 +27,10 @@ my $optional = 0; my $need_symlink = 0; my $need_sphinx = 0; my $install = ""; +my $min_version; +my $rec_version; +my $cur_version; +my $virtenv_dir = "sphinx_"; # # Command line arguments @@ -76,6 +80,52 @@ my %texlive = ( # Subroutines that checks if a feature exists # +sub handle_sphinx_version() +{ + open IN, $conf; + while (<IN>) { + if (m/^\s*needs_sphinx\s*=\s*[\'\"]([\d\.]+)[\'\"]/) { + $min_version=$1; + last; + } + } + close IN; + + die "Can't get needs_sphinx version from $conf" if (!$min_version); + + open IN, $requirement_file; + while (<IN>) { + if (m/^\s*Sphinx\s*==\s*([\d\.]+)$/) { + $rec_version=$1; + last; + } + } + close IN; + + open IN, "sphinx-build --version 2>&1 |"; + while (<IN>) { + if (m/^\s*sphinx-build\s+([\d\.]+)$/) { + $cur_version=$1; + last; + } + } + close IN; + + $virtenv_dir .= $rec_version; + + # Sphinx is not installed + return if (!$cur_version); + + if ($cur_version lt $min_version) { + print "Sphinx version older than $min_version! We recommend at least $rec_version"; + exit -1; + } + + if ($cur_version lt $rec_version) { + print "Warning: we recommend at least Sphinx version $rec_version"; + } +} + sub check_missing(%) { my %map = %{$_[0]}; @@ -587,6 +637,8 @@ while (@ARGV) { } } +handle_sphinx_version(); + # # Determine the system type. There's no standard unique way that would # work with all distros with a minimal package install. So, several