From patchwork Wed Apr 3 22:26:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Cole Robinson X-Patchwork-Id: 161716 Delivered-To: patch@linaro.org Received: by 2002:a02:c6d8:0:0:0:0:0 with SMTP id r24csp862243jan; Wed, 3 Apr 2019 15:27:04 -0700 (PDT) X-Google-Smtp-Source: APXvYqyUdtWM6cmSc0QtE9Dlso3ahjBqEeFD1+/QH/TJFVRcS9TbdatR+kp+lCvQIZg3mVqwy3Wj X-Received: by 2002:aed:2a3b:: with SMTP id c56mr2300364qtd.110.1554330424885; Wed, 03 Apr 2019 15:27:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1554330424; cv=none; d=google.com; s=arc-20160816; b=Ytniezt4qbQdaSMSGvSMt0NvDc9GjDhzXjYBFWRtApcNDlnMQmIJCmGukMY9SKJuNh OcV8TivHdPdUh/wNA89cVzu/L8B0I/E6LhPnGS8WGZXrvetB3E8tKtBOa9bJLHBaGRMO +Q1j888T2ebdlzQrK7Ey4WWD/6sKq4nUSt3ItAn6MzaqZd44ibpcdsNg9CGBMngqlxRG TxAw6/jDoh3yxKF1AEMW3FLWZVwTdeyQi+/mQFkeaxxUYHJ5pt5qW1/xEJ6h7iJf/OAy pS6EmNUBjY5hwWTltNNaW+N3tze81OwXW0Foe/Aaox4Bvia0BEaQ45lNKgwHioAQB8vI XYsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:sender:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :delivered-to; bh=kKMvP+L4P/Ph6L4iBTCI7ric2IM71MjfIk+w/5glm/0=; b=Xmq0lMy4+czsBD20osNVit1cbGJMw50s6kymw2gQchhcj8kA8BkJMm4H16d22aMybe cYPfwWN2TER6DMyiDDPPppNbY9sRTJJ+Ol5561Pe1dLIuf/S64qymYtswQO/55uihK0f YDn12CC+g6nDeOJMja9QK+mc7A1UsjaX5PSVsdE2O6NpcgI5Rv559zZ67zwmU0kEu1ja 0teCJgz9yKILRlhr14wZoiyf0cuBHwk05WCjFFbWZM83nwAjR0BFOH3z21j1j6uacUwh hVAveQRgUsCa4EwYE1EHJ19MErR5yg7FxR3phh8smxt9ahoR++aYJ8N7pvTDO7+kif1d orYA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of libvir-list-bounces@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id r42si6138869qtk.226.2019.04.03.15.27.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Apr 2019 15:27:04 -0700 (PDT) Received-SPF: pass (google.com: domain of libvir-list-bounces@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of libvir-list-bounces@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6BF44307E05C; Wed, 3 Apr 2019 22:27:03 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 453E019C69; Wed, 3 Apr 2019 22:27:03 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 971FC41F3D; Wed, 3 Apr 2019 22:27:01 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x33MQw5k020283 for ; Wed, 3 Apr 2019 18:26:58 -0400 Received: by smtp.corp.redhat.com (Postfix) id D7C9510A33C2; Wed, 3 Apr 2019 22:26:58 +0000 (UTC) Delivered-To: libvirt-list@redhat.com Received: from worklaptop.redhat.com (ovpn-120-38.rdu2.redhat.com [10.10.120.38]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6D87C1001DDD; Wed, 3 Apr 2019 22:26:58 +0000 (UTC) From: Cole Robinson To: libvirt-list@redhat.com Date: Wed, 3 Apr 2019 18:26:49 -0400 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH 1/3] docs: Use google sitesearch for website search X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.42]); Wed, 03 Apr 2019 22:27:03 +0000 (UTC) The website search is perpetually broken, has had XSS issues in the past, and I suspect when it's working it's not as fast or capable as a simple google site:libvirt.org search Replace the
implementation with one that sends the user to google.com with 'site:libvirt.org' appended to the search string Signed-off-by: Cole Robinson --- docs/page.xsl | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) -- 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list Reviewed-by: Daniel P. Berrangé diff --git a/docs/page.xsl b/docs/page.xsl index 4698e2789e..3d007f486c 100644 --- a/docs/page.xsl +++ b/docs/page.xsl @@ -155,11 +155,10 @@ From patchwork Wed Apr 3 22:26:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Cole Robinson X-Patchwork-Id: 161718 Delivered-To: patch@linaro.org Received: by 2002:a02:c6d8:0:0:0:0:0 with SMTP id r24csp862372jan; Wed, 3 Apr 2019 15:27:16 -0700 (PDT) X-Google-Smtp-Source: APXvYqzA5R6fq+q4U/0o56CegOBUc8n+FwQxdxXErW8/a6CCtBFkDILbiserJ9EI/Va50c8Z+R74 X-Received: by 2002:a0c:9666:: with SMTP id 35mr1977561qvy.30.1554330436266; Wed, 03 Apr 2019 15:27:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1554330436; cv=none; d=google.com; s=arc-20160816; b=wy808nCNw6AIowjy+9PKHI+ttUpbvF3UPB57ntQRyxUnQEJnWanfbMHFnZGA6XvFqi JznHPropLLTMEaUSkpXDKCzItjm6ik6K3+kQLifx3yBT5BeLnk/Z8PuQCCRMYEDrA8T7 GUvZl/AGRtaSrju7HGNc/mPxUhWlXGC7prQP/o9686o7YMFn4CHiKqA2LdtWyzBrwv4D u0oXns6k9mFAb4Tc4hkqMc6W5jF9vCvTUdo0SQUNtrTwwqE0eesIuRyrp8XCRGlz9jJm 8gA9OJrnlmhVAyZS/8eyMAhXmjo7NRjPNLSLtDTLLZzxiY/dAD6OLCZuid1LGg5M2Ywc WPKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:sender:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :delivered-to; bh=/r/t2NG6xnlY9QrNSoRNQ1h5sgIqhPgnPyJzjHJdMaw=; b=Po3Uo+KF0JX27ahlTqAY1mJ9dIJl+eik/GQquAj3A/GCtXDlEDQ2DMa6lk4TZ6hjpQ TT1wTPPJOzVfinkPQhpMWZjPDfo10JtvI2D2dATIrtSfP3y75BZ02HbL8iJvCH/PHz7K S/AlrQtQlJWYde3NRjs2A9i00+ybtORYuRQxlrxXrnYym3eVJclx79tDWRg+KZXoFkJS ozUYOxPrKymn/OnoULfrk3C9WOZN3jmLC2uZahEJNUDNVwyUsyaimZWdK6zS35+/ENro kMi05jJrmqu693bfQGGxWwuvIENwGSrt7LH5Udkzti85gQBMZ6vnEw6VKMCl3OeBNYAr 50CQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of libvir-list-bounces@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id t21si7994421qte.338.2019.04.03.15.27.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Apr 2019 15:27:16 -0700 (PDT) Received-SPF: pass (google.com: domain of libvir-list-bounces@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of libvir-list-bounces@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DB2B53164680; Wed, 3 Apr 2019 22:27:14 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B4BB8179C9; Wed, 3 Apr 2019 22:27:14 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 7755B181AC46; Wed, 3 Apr 2019 22:27:14 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x33MQxBp020291 for ; Wed, 3 Apr 2019 18:26:59 -0400 Received: by smtp.corp.redhat.com (Postfix) id 9239310A33C4; Wed, 3 Apr 2019 22:26:59 +0000 (UTC) Delivered-To: libvirt-list@redhat.com Received: from worklaptop.redhat.com (ovpn-120-38.rdu2.redhat.com [10.10.120.38]) by smtp.corp.redhat.com (Postfix) with ESMTP id 123D410A33C1; Wed, 3 Apr 2019 22:26:58 +0000 (UTC) From: Cole Robinson To: libvirt-list@redhat.com Date: Wed, 3 Apr 2019 18:26:50 -0400 Message-Id: <5a008deb0273da0274c1b0b0f8a89e36887fff00.1554329992.git.crobinso@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH 2/3] docs: Remove index.py X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]); Wed, 03 Apr 2019 22:27:15 +0000 (UTC) This was used for generating the website search, which now just calls out to google. Remove it Signed-off-by: Cole Robinson --- docs/index.py | 1266 ------------------------------------------------- 1 file changed, 1266 deletions(-) delete mode 100755 docs/index.py -- 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list Reviewed-by: Daniel P. Berrangé diff --git a/docs/index.py b/docs/index.py deleted file mode 100755 index 0d07ca4d05..0000000000 --- a/docs/index.py +++ /dev/null @@ -1,1266 +0,0 @@ -#!/usr/bin/env python2 -# -# imports the API description and fills up a database with -# name relevance to modules, functions or web pages -# -# Operation needed: -# ================= -# -# install mysqld, the python wrappers for mysql and libxml2, start mysqld -# - mysql-server -# - mysql -# - php-mysql -# - MySQL-python -# Change the root passwd of mysql: -# mysqladmin -u root password new_password -# Create the new database libvir -# mysqladmin -p create libvir -# Create a database user 'veillard' and give him password access -# change veillard and abcde with the right user name and passwd -# mysql -p -# password: -# mysql> GRANT ALL PRIVILEGES ON libvir TO veillard@localhost -# IDENTIFIED BY 'abcde' WITH GRANT OPTION; -# mysql> GRANT ALL PRIVILEGES ON libvir.* TO veillard@localhost -# IDENTIFIED BY 'abcde' WITH GRANT OPTION; -# -# As the user check the access: -# mysql -p libvir -# Enter password: -# Welcome to the MySQL monitor.... -# mysql> use libvir -# Database changed -# mysql> quit -# Bye -# -# Then run the script in the doc subdir, it will create the symbols and -# word tables and populate them with information extracted from -# the libvirt-api.xml API description, and make then accessible read-only -# by nobody@loaclhost the user expected to be Apache's one -# -# On the Apache configuration, make sure you have php support enabled -# - -import MySQLdb -import libxml2 -import sys -import string -import os - -# -# We are not interested in parsing errors here -# -def callback(ctx, str): - return -libxml2.registerErrorHandler(callback, None) - -# -# The dictionary of tables required and the SQL command needed -# to create them -# -TABLES = { - "symbols": """CREATE TABLE symbols ( - name varchar(255) BINARY NOT NULL, - module varchar(255) BINARY NOT NULL, - type varchar(25) NOT NULL, - descr varchar(255), - UNIQUE KEY name (name), - KEY module (module))""", - "words": """CREATE TABLE words ( - name varchar(50) BINARY NOT NULL, - symbol varchar(255) BINARY NOT NULL, - relevance int, - KEY name (name), - KEY symbol (symbol), - UNIQUE KEY ID (name, symbol))""", - "wordsHTML": """CREATE TABLE wordsHTML ( - name varchar(50) BINARY NOT NULL, - resource varchar(255) BINARY NOT NULL, - section varchar(255), - id varchar(50), - relevance int, - KEY name (name), - KEY resource (resource), - UNIQUE KEY ref (name, resource))""", - "wordsArchive": """CREATE TABLE wordsArchive ( - name varchar(50) BINARY NOT NULL, - ID int(11) NOT NULL, - relevance int, - KEY name (name), - UNIQUE KEY ref (name, ID))""", - "pages": """CREATE TABLE pages ( - resource varchar(255) BINARY NOT NULL, - title varchar(255) BINARY NOT NULL, - UNIQUE KEY name (resource))""", - "archives": """CREATE TABLE archives ( - ID int(11) NOT NULL auto_increment, - resource varchar(255) BINARY NOT NULL, - title varchar(255) BINARY NOT NULL, - UNIQUE KEY id (ID,resource(255)), - INDEX (ID), - INDEX (resource))""", - "Queries": """CREATE TABLE Queries ( - ID int(11) NOT NULL auto_increment, - Value varchar(50) NOT NULL, - Count int(11) NOT NULL, - UNIQUE KEY id (ID,Value(35)), - INDEX (ID))""", - "AllQueries": """CREATE TABLE AllQueries ( - ID int(11) NOT NULL auto_increment, - Value varchar(50) NOT NULL, - Count int(11) NOT NULL, - UNIQUE KEY id (ID,Value(35)), - INDEX (ID))""", -} - -# -# The XML API description file to parse -# -API = "libvirt-api.xml" -DB = None - -######################################################################### -# # -# MySQL database interfaces # -# # -######################################################################### -def createTable(db, name): - global TABLES - - if db is None: - return -1 - if name is None: - return -1 - c = db.cursor() - - ret = c.execute("DROP TABLE IF EXISTS %s" % (name)) - if ret == 1: - print "Removed table %s" % (name) - print "Creating table %s" % (name) - try: - ret = c.execute(TABLES[name]) - except: - print "Failed to create table %s" % (name) - return -1 - return ret - -def checkTables(db, verbose=1): - global TABLES - - if db is None: - return -1 - c = db.cursor() - nbtables = c.execute("show tables") - if verbose: - print "Found %d tables" % (nbtables) - tables = {} - i = 0 - while i < nbtables: - l = c.fetchone() - name = l[0] - tables[name] = {} - i = i + 1 - - for table in TABLES.keys(): - if not tables.has_key(table): - print "table %s missing" % (table) - createTable(db, table) - try: - ret = c.execute("SELECT count(*) from %s" % table) - row = c.fetchone() - if verbose: - print "Table %s contains %d records" % (table, row[0]) - except: - print "Troubles with table %s: repairing" % (table) - ret = c.execute("repair table %s" % table) - print "repairing returned %d" % (ret) - ret = c.execute("SELECT count(*) from %s" % table) - row = c.fetchone() - print "Table %s contains %d records" % (table, row[0]) - if verbose: - print "checkTables finished" - - # make sure apache can access the tables read-only - try: - ret = c.execute("GRANT SELECT ON libvir.* TO nobody@localhost") - ret = c.execute("GRANT INSERT,SELECT,UPDATE ON libvir.Queries TO nobody@localhost") - except: - pass - return 0 - -def openMySQL(db="libvir", passwd=None, verbose=1): - global DB - - if passwd is None: - try: - passwd = os.environ["MySQL_PASS"] - except: - print "No password available, set environment MySQL_PASS" - sys.exit(1) - - DB = MySQLdb.connect(passwd=passwd, db=db) - if DB is None: - return -1 - ret = checkTables(DB, verbose) - return ret - -def updateWord(name, symbol, relevance): - global DB - - if DB is None: - openMySQL() - if DB is None: - return -1 - if name is None: - return -1 - if symbol is None: - return -1 - - c = DB.cursor() - try: - ret = c.execute( -"""INSERT INTO words (name, symbol, relevance) VALUES ('%s','%s', %d)""" % - (name, symbol, relevance)) - except: - try: - ret = c.execute( - """UPDATE words SET relevance = %d where name = '%s' and symbol = '%s'""" % - (relevance, name, symbol)) - except: - print "Update word (%s, %s, %s) failed command" % (name, symbol, relevance) - print "UPDATE words SET relevance = %d where name = '%s' and symbol = '%s'" % (relevance, name, symbol) - print sys.exc_type, sys.exc_value - return -1 - - return ret - -def updateSymbol(name, module, type, desc): - global DB - - updateWord(name, name, 50) - if DB is None: - openMySQL() - if DB is None: - return -1 - if name is None: - return -1 - if module is None: - return -1 - if type is None: - return -1 - - try: - desc = string.replace(desc, "'", " ") - l = string.split(desc, ".") - desc = l[0] - desc = desc[0:99] - except: - desc = "" - - c = DB.cursor() - try: - ret = c.execute( -"""INSERT INTO symbols (name, module, type, descr) VALUES ('%s','%s', '%s', '%s')""" % - (name, module, type, desc)) - except: - try: - ret = c.execute( -"""UPDATE symbols SET module='%s', type='%s', descr='%s' where name='%s'""" % - (module, type, desc, name)) - except: - print "Update symbol (%s, %s, %s) failed command" % (name, module, type) - print """UPDATE symbols SET module='%s', type='%s', descr='%s' where name='%s'""" % (module, type, desc, name) - print sys.exc_type, sys.exc_value - return -1 - - return ret - -def addFunction(name, module, desc=""): - return updateSymbol(name, module, 'function', desc) - -def addMacro(name, module, desc=""): - return updateSymbol(name, module, 'macro', desc) - -def addEnum(name, module, desc=""): - return updateSymbol(name, module, 'enum', desc) - -def addStruct(name, module, desc=""): - return updateSymbol(name, module, 'struct', desc) - -def addConst(name, module, desc=""): - return updateSymbol(name, module, 'const', desc) - -def addType(name, module, desc=""): - return updateSymbol(name, module, 'type', desc) - -def addFunctype(name, module, desc=""): - return updateSymbol(name, module, 'functype', desc) - -def addPage(resource, title): - global DB - - if DB is None: - openMySQL() - if DB is None: - return -1 - if resource is None: - return -1 - - c = DB.cursor() - try: - ret = c.execute( - """INSERT INTO pages (resource, title) VALUES ('%s','%s')""" % - (resource, title)) - except: - try: - ret = c.execute( - """UPDATE pages SET title='%s' WHERE resource='%s'""" % - (title, resource)) - except: - print "Update symbol (%s, %s, %s) failed command" % (name, module, type) - print """UPDATE pages SET title='%s' WHERE resource='%s'""" % (title, resource) - print sys.exc_type, sys.exc_value - return -1 - - return ret - -def updateWordHTML(name, resource, desc, id, relevance): - global DB - - if DB is None: - openMySQL() - if DB is None: - return -1 - if name is None: - return -1 - if resource is None: - return -1 - if id is None: - id = "" - if desc is None: - desc = "" - else: - try: - desc = string.replace(desc, "'", " ") - desc = desc[0:99] - except: - desc = "" - - c = DB.cursor() - try: - ret = c.execute( -"""INSERT INTO wordsHTML (name, resource, section, id, relevance) VALUES ('%s','%s', '%s', '%s', '%d')""" % - (name, resource, desc, id, relevance)) - except: - try: - ret = c.execute( -"""UPDATE wordsHTML SET section='%s', id='%s', relevance='%d' where name='%s' and resource='%s'""" % - (desc, id, relevance, name, resource)) - except: - print "Update symbol (%s, %s, %d) failed command" % (name, resource, relevance) - print """UPDATE wordsHTML SET section='%s', id='%s', relevance='%d' where name='%s' and resource='%s'""" % (desc, id, relevance, name, resource) - print sys.exc_type, sys.exc_value - return -1 - - return ret - -def checkXMLMsgArchive(url): - global DB - - if DB is None: - openMySQL() - if DB is None: - return -1 - if url is None: - return -1 - - c = DB.cursor() - try: - ret = c.execute( - """SELECT ID FROM archives WHERE resource='%s'""" % (url)) - row = c.fetchone() - if row is None: - return -1 - except: - return -1 - - return row[0] - -def addXMLMsgArchive(url, title): - global DB - - if DB is None: - openMySQL() - if DB is None: - return -1 - if url is None: - return -1 - if title is None: - title = "" - else: - title = string.replace(title, "'", " ") - title = title[0:99] - - c = DB.cursor() - try: - cmd = """INSERT INTO archives (resource, title) VALUES ('%s','%s')""" % (url, title) - ret = c.execute(cmd) - cmd = """SELECT ID FROM archives WHERE resource='%s'""" % (url) - ret = c.execute(cmd) - row = c.fetchone() - if row is None: - print "addXMLMsgArchive failed to get the ID: %s" % (url) - return -1 - except: - print "addXMLMsgArchive failed command: %s" % (cmd) - return -1 - - return((int)(row[0])) - -def updateWordArchive(name, id, relevance): - global DB - - if DB is None: - openMySQL() - if DB is None: - return -1 - if name is None: - return -1 - if id is None: - return -1 - - c = DB.cursor() - try: - ret = c.execute( -"""INSERT INTO wordsArchive (name, id, relevance) VALUES ('%s', '%d', '%d')""" % - (name, id, relevance)) - except: - try: - ret = c.execute( -"""UPDATE wordsArchive SET relevance='%d' where name='%s' and ID='%d'""" % - (relevance, name, id)) - except: - print "Update word archive (%s, %d, %d) failed command" % (name, id, relevance) - print """UPDATE wordsArchive SET relevance='%d' where name='%s' and ID='%d'""" % (relevance, name, id) - print sys.exc_type, sys.exc_value - return -1 - - return ret - -######################################################################### -# # -# Word dictionary and analysis routines # -# # -######################################################################### - -# -# top 100 english word without the one len < 3 + own set -# -dropWords = { - 'the':0, 'this':0, 'can':0, 'man':0, 'had':0, 'him':0, 'only':0, - 'and':0, 'not':0, 'been':0, 'other':0, 'even':0, 'are':0, 'was':0, - 'new':0, 'most':0, 'but':0, 'when':0, 'some':0, 'made':0, 'from':0, - 'who':0, 'could':0, 'after':0, 'that':0, 'will':0, 'time':0, 'also':0, - 'have':0, 'more':0, 'these':0, 'did':0, 'was':0, 'two':0, 'many':0, - 'they':0, 'may':0, 'before':0, 'for':0, 'which':0, 'out':0, 'then':0, - 'must':0, 'one':0, 'through':0, 'with':0, 'you':0, 'said':0, - 'first':0, 'back':0, 'were':0, 'what':0, 'any':0, 'years':0, 'his':0, - 'her':0, 'where':0, 'all':0, 'its':0, 'now':0, 'much':0, 'she':0, - 'about':0, 'such':0, 'your':0, 'there':0, 'into':0, 'like':0, 'may':0, - 'would':0, 'than':0, 'our':0, 'well':0, 'their':0, 'them':0, 'over':0, - 'down':0, - 'net':0, 'www':0, 'bad':0, 'Okay':0, 'bin':0, 'cur':0, -} - -wordsDict = {} -wordsDictHTML = {} -wordsDictArchive = {} - -def cleanupWordsString(str): - str = string.replace(str, ".", " ") - str = string.replace(str, "!", " ") - str = string.replace(str, "?", " ") - str = string.replace(str, ",", " ") - str = string.replace(str, "'", " ") - str = string.replace(str, '"', " ") - str = string.replace(str, ";", " ") - str = string.replace(str, "(", " ") - str = string.replace(str, ")", " ") - str = string.replace(str, "{", " ") - str = string.replace(str, "}", " ") - str = string.replace(str, "<", " ") - str = string.replace(str, ">", " ") - str = string.replace(str, "=", " ") - str = string.replace(str, "/", " ") - str = string.replace(str, "*", " ") - str = string.replace(str, ":", " ") - str = string.replace(str, "#", " ") - str = string.replace(str, "\\", " ") - str = string.replace(str, "\n", " ") - str = string.replace(str, "\r", " ") - str = string.replace(str, "\xc2", " ") - str = string.replace(str, "\xa0", " ") - return str - -def cleanupDescrString(str): - str = string.replace(str, "'", " ") - str = string.replace(str, "\n", " ") - str = string.replace(str, "\r", " ") - str = string.replace(str, "\xc2", " ") - str = string.replace(str, "\xa0", " ") - l = string.split(str) - str = string.join(str) - return str - -def splitIdentifier(str): - ret = [] - while str != "": - cur = string.lower(str[0]) - str = str[1:] - if ((cur < 'a') or (cur > 'z')): - continue - while (str != "") and (str[0] >= 'A') and (str[0] <= 'Z'): - cur = cur + string.lower(str[0]) - str = str[1:] - while (str != "") and (str[0] >= 'a') and (str[0] <= 'z'): - cur = cur + str[0] - str = str[1:] - while (str != "") and (str[0] >= '0') and (str[0] <= '9'): - str = str[1:] - ret.append(cur) - return ret - -def addWord(word, module, symbol, relevance): - global wordsDict - - if word is None or len(word) < 3: - return -1 - if module is None or symbol is None: - return -1 - if dropWords.has_key(word): - return 0 - if ord(word[0]) > 0x80: - return 0 - - if wordsDict.has_key(word): - d = wordsDict[word] - if d is None: - return 0 - if len(d) > 500: - wordsDict[word] = None - return 0 - try: - relevance = relevance + d[(module, symbol)] - except: - pass - else: - wordsDict[word] = {} - wordsDict[word][(module, symbol)] = relevance - return relevance - -def addString(str, module, symbol, relevance): - if str is None or len(str) < 3: - return -1 - ret = 0 - str = cleanupWordsString(str) - l = string.split(str) - for word in l: - if len(word) > 2: - ret = ret + addWord(word, module, symbol, 5) - - return ret - -def addWordHTML(word, resource, id, section, relevance): - global wordsDictHTML - - if word is None or len(word) < 3: - return -1 - if resource is None or section is None: - return -1 - if dropWords.has_key(word): - return 0 - if ord(word[0]) > 0x80: - return 0 - - section = cleanupDescrString(section) - - if wordsDictHTML.has_key(word): - d = wordsDictHTML[word] - if d is None: - print "skipped %s" % (word) - return 0 - try: - (r,i,s) = d[resource] - if i is not None: - id = i - if s is not None: - section = s - relevance = relevance + r - except: - pass - else: - wordsDictHTML[word] = {} - d = wordsDictHTML[word] - d[resource] = (relevance, id, section) - return relevance - -def addStringHTML(str, resource, id, section, relevance): - if str is None or len(str) < 3: - return -1 - ret = 0 - str = cleanupWordsString(str) - l = string.split(str) - for word in l: - if len(word) > 2: - try: - r = addWordHTML(word, resource, id, section, relevance) - if r < 0: - print "addWordHTML failed: %s %s" % (word, resource) - ret = ret + r - except: - print "addWordHTML failed: %s %s %d" % (word, resource, relevance) - print sys.exc_type, sys.exc_value - - return ret - -def addWordArchive(word, id, relevance): - global wordsDictArchive - - if word is None or len(word) < 3: - return -1 - if id is None or id == -1: - return -1 - if dropWords.has_key(word): - return 0 - if ord(word[0]) > 0x80: - return 0 - - if wordsDictArchive.has_key(word): - d = wordsDictArchive[word] - if d is None: - print "skipped %s" % (word) - return 0 - try: - r = d[id] - relevance = relevance + r - except: - pass - else: - wordsDictArchive[word] = {} - d = wordsDictArchive[word] - d[id] = relevance - return relevance - -def addStringArchive(str, id, relevance): - if str is None or len(str) < 3: - return -1 - ret = 0 - str = cleanupWordsString(str) - l = string.split(str) - for word in l: - i = len(word) - if i > 2: - try: - r = addWordArchive(word, id, relevance) - if r < 0: - print "addWordArchive failed: %s %s" % (word, id) - else: - ret = ret + r - except: - print "addWordArchive failed: %s %s %d" % (word, id, relevance) - print sys.exc_type, sys.exc_value - return ret - -######################################################################### -# # -# XML API description analysis # -# # -######################################################################### - -def loadAPI(filename): - doc = libxml2.parseFile(filename) - print "loaded %s" % (filename) - return doc - -def foundExport(file, symbol): - if file is None: - return 0 - if symbol is None: - return 0 - addFunction(symbol, file) - l = splitIdentifier(symbol) - for word in l: - addWord(word, file, symbol, 10) - return 1 - -def analyzeAPIFile(top): - count = 0 - name = top.prop("name") - cur = top.children - while cur is not None: - if cur.type == 'text': - cur = cur.next - continue - if cur.name == "exports": - count = count + foundExport(name, cur.prop("symbol")) - else: - print "unexpected element %s in API doc " % (name) - cur = cur.next - return count - -def analyzeAPIFiles(top): - count = 0 - cur = top.children - - while cur is not None: - if cur.type == 'text': - cur = cur.next - continue - if cur.name == "file": - count = count + analyzeAPIFile(cur) - else: - print "unexpected element %s in API doc " % (cur.name) - cur = cur.next - return count - -def analyzeAPIEnum(top): - file = top.prop("file") - if file is None: - return 0 - symbol = top.prop("name") - if symbol is None: - return 0 - - addEnum(symbol, file) - l = splitIdentifier(symbol) - for word in l: - addWord(word, file, symbol, 10) - - return 1 - -def analyzeAPIConst(top): - file = top.prop("file") - if file is None: - return 0 - symbol = top.prop("name") - if symbol is None: - return 0 - - addConst(symbol, file) - l = splitIdentifier(symbol) - for word in l: - addWord(word, file, symbol, 10) - - return 1 - -def analyzeAPIType(top): - file = top.prop("file") - if file is None: - return 0 - symbol = top.prop("name") - if symbol is None: - return 0 - - addType(symbol, file) - l = splitIdentifier(symbol) - for word in l: - addWord(word, file, symbol, 10) - return 1 - -def analyzeAPIFunctype(top): - file = top.prop("file") - if file is None: - return 0 - symbol = top.prop("name") - if symbol is None: - return 0 - - addFunctype(symbol, file) - l = splitIdentifier(symbol) - for word in l: - addWord(word, file, symbol, 10) - return 1 - -def analyzeAPIStruct(top): - file = top.prop("file") - if file is None: - return 0 - symbol = top.prop("name") - if symbol is None: - return 0 - - addStruct(symbol, file) - l = splitIdentifier(symbol) - for word in l: - addWord(word, file, symbol, 10) - - info = top.prop("info") - if info is not None: - info = string.replace(info, "'", " ") - info = string.strip(info) - l = string.split(info) - for word in l: - if len(word) > 2: - addWord(word, file, symbol, 5) - return 1 - -def analyzeAPIMacro(top): - file = top.prop("file") - if file is None: - return 0 - symbol = top.prop("name") - if symbol is None: - return 0 - symbol = string.replace(symbol, "'", " ") - symbol = string.strip(symbol) - - info = None - cur = top.children - while cur is not None: - if cur.type == 'text': - cur = cur.next - continue - if cur.name == "info": - info = cur.content - break - cur = cur.next - - l = splitIdentifier(symbol) - for word in l: - addWord(word, file, symbol, 10) - - if info is None: - addMacro(symbol, file) - print "Macro %s description has no " % (symbol) - return 0 - - info = string.replace(info, "'", " ") - info = string.strip(info) - addMacro(symbol, file, info) - l = string.split(info) - for word in l: - if len(word) > 2: - addWord(word, file, symbol, 5) - return 1 - -def analyzeAPIFunction(top): - file = top.prop("file") - if file is None: - return 0 - symbol = top.prop("name") - if symbol is None: - return 0 - - symbol = string.replace(symbol, "'", " ") - symbol = string.strip(symbol) - info = None - cur = top.children - while cur is not None: - if cur.type == 'text': - cur = cur.next - continue - if cur.name == "info": - info = cur.content - elif cur.name == "return": - rinfo = cur.prop("info") - if rinfo is not None: - rinfo = string.replace(rinfo, "'", " ") - rinfo = string.strip(rinfo) - addString(rinfo, file, symbol, 7) - elif cur.name == "arg": - ainfo = cur.prop("info") - if ainfo is not None: - ainfo = string.replace(ainfo, "'", " ") - ainfo = string.strip(ainfo) - addString(ainfo, file, symbol, 5) - name = cur.prop("name") - if name is not None: - name = string.replace(name, "'", " ") - name = string.strip(name) - addWord(name, file, symbol, 7) - cur = cur.next - if info is None: - print "Function %s description has no " % (symbol) - addFunction(symbol, file, "") - else: - info = string.replace(info, "'", " ") - info = string.strip(info) - addFunction(symbol, file, info) - addString(info, file, symbol, 5) - - l = splitIdentifier(symbol) - for word in l: - addWord(word, file, symbol, 10) - - return 1 - -def analyzeAPISymbols(top): - count = 0 - cur = top.children - - while cur is not None: - if cur.type == 'text': - cur = cur.next - continue - if cur.name == "macro": - count = count + analyzeAPIMacro(cur) - elif cur.name == "function": - count = count + analyzeAPIFunction(cur) - elif cur.name == "const": - count = count + analyzeAPIConst(cur) - elif cur.name == "typedef": - count = count + analyzeAPIType(cur) - elif cur.name == "struct": - count = count + analyzeAPIStruct(cur) - elif cur.name == "enum": - count = count + analyzeAPIEnum(cur) - elif cur.name == "functype": - count = count + analyzeAPIFunctype(cur) - else: - print "unexpected element %s in API doc " % (cur.name) - cur = cur.next - return count - -def analyzeAPI(doc): - count = 0 - if doc is None: - return -1 - root = doc.getRootElement() - if root.name != "api": - print "Unexpected root name" - return -1 - cur = root.children - while cur is not None: - if cur.type == 'text': - cur = cur.next - continue - if cur.name == "files": - pass -# count = count + analyzeAPIFiles(cur) - elif cur.name == "symbols": - count = count + analyzeAPISymbols(cur) - else: - print "unexpected element %s in API doc" % (cur.name) - cur = cur.next - return count - -######################################################################### -# # -# Web pages parsing and analysis # -# # -######################################################################### - -import glob - -def analyzeHTMLText(doc, resource, p, section, id): - words = 0 - try: - content = p.content - words = words + addStringHTML(content, resource, id, section, 5) - except: - return -1 - return words - -def analyzeHTMLPara(doc, resource, p, section, id): - words = 0 - try: - content = p.content - words = words + addStringHTML(content, resource, id, section, 5) - except: - return -1 - return words - -def analyzeHTMLPre(doc, resource, p, section, id): - words = 0 - try: - content = p.content - words = words + addStringHTML(content, resource, id, section, 5) - except: - return -1 - return words - -def analyzeHTML(doc, resource, p, section, id): - words = 0 - try: - content = p.content - words = words + addStringHTML(content, resource, id, section, 5) - except: - return -1 - return words - -def analyzeHTML(doc, resource): - para = 0 - ctxt = doc.xpathNewContext() - try: - res = ctxt.xpathEval("//head/title") - title = res[0].content - except: - title = "Page %s" % (resource) - addPage(resource, title) - try: - items = ctxt.xpathEval("//h1 | //h2 | //h3 | //text()") - section = title - id = "" - for item in items: - if item.name == 'h1' or item.name == 'h2' or item.name == 'h3': - section = item.content - if item.prop("id"): - id = item.prop("id") - elif item.prop("name"): - id = item.prop("name") - elif item.type == 'text': - analyzeHTMLText(doc, resource, item, section, id) - para = para + 1 - elif item.name == 'p': - analyzeHTMLPara(doc, resource, item, section, id) - para = para + 1 - elif item.name == 'pre': - analyzeHTMLPre(doc, resource, item, section, id) - para = para + 1 - else: - print "Page %s, unexpected %s element" % (resource, item.name) - except: - print "Page %s: problem analyzing" % (resource) - print sys.exc_type, sys.exc_value - - return para - -def analyzeHTMLPages(): - ret = 0 - HTMLfiles = glob.glob("*.html") + glob.glob("tutorial/*.html") + \ - glob.glob("CIM/*.html") + glob.glob("ocaml/*.html") + \ - glob.glob("ruby/*.html") - for html in HTMLfiles: - if html[0:3] == "API": - continue - if html == "xml.html": - continue - try: - doc = libxml2.parseFile(html) - except: - doc = libxml2.htmlParseFile(html, None) - try: - res = analyzeHTML(doc, html) - print "Parsed %s: %d paragraphs" % (html, res) - ret = ret + 1 - except: - print "could not parse %s" % (html) - return ret - -######################################################################### -# # -# Mail archives parsing and analysis # -# # -######################################################################### - -import time - -def getXMLDateArchive(t=None): - if t is None: - t = time.time() - T = time.gmtime(t) - month = time.strftime("%B", T) - year = T[0] - url = "http://www.redhat.com/archives/libvir-list/%d-%s/date.html" % (year, month) - return url - -def scanXMLMsgArchive(url, title, force=0): - if url is None or title is None: - return 0 - - ID = checkXMLMsgArchive(url) - if force == 0 and ID != -1: - return 0 - - if ID == -1: - ID = addXMLMsgArchive(url, title) - if ID == -1: - return 0 - - try: - print "Loading %s" % (url) - doc = libxml2.htmlParseFile(url, None) - except: - doc = None - if doc is None: - print "Failed to parse %s" % (url) - return 0 - - addStringArchive(title, ID, 20) - ctxt = doc.xpathNewContext() - texts = ctxt.xpathEval("//pre//text()") - for text in texts: - addStringArchive(text.content, ID, 5) - - return 1 - -def scanXMLDateArchive(t=None, force=0): - global wordsDictArchive - - wordsDictArchive = {} - - url = getXMLDateArchive(t) - print "loading %s" % (url) - try: - doc = libxml2.htmlParseFile(url, None) - except: - doc = None - if doc is None: - print "Failed to parse %s" % (url) - return -1 - ctxt = doc.xpathNewContext() - anchors = ctxt.xpathEval("//a[@href]") - links = 0 - newmsg = 0 - for anchor in anchors: - href = anchor.prop("href") - if href is None or href[0:3] != "msg": - continue - try: - links = links + 1 - - msg = libxml2.buildURI(href, url) - title = anchor.content - if title is not None and title[0:4] == 'Re: ': - title = title[4:] - if title is not None and title[0:6] == '[xml] ': - title = title[6:] - newmsg = newmsg + scanXMLMsgArchive(msg, title, force) - - except: - pass - - return newmsg - - -######################################################################### -# # -# Main code: open the DB, the API XML and analyze it # -# # -######################################################################### -def analyzeArchives(t=None, force=0): - global wordsDictArchive - - ret = scanXMLDateArchive(t, force) - print "Indexed %d words in %d archive pages" % (len(wordsDictArchive), ret) - - i = 0 - skipped = 0 - for word in wordsDictArchive.keys(): - refs = wordsDictArchive[word] - if refs is None: - skipped = skipped + 1 - continue - for id in refs.keys(): - relevance = refs[id] - updateWordArchive(word, id, relevance) - i = i + 1 - - print "Found %d associations in HTML pages" % (i) - -def analyzeHTMLTop(): - global wordsDictHTML - - ret = analyzeHTMLPages() - print "Indexed %d words in %d HTML pages" % (len(wordsDictHTML), ret) - - i = 0 - skipped = 0 - for word in wordsDictHTML.keys(): - refs = wordsDictHTML[word] - if refs is None: - skipped = skipped + 1 - continue - for resource in refs.keys(): - (relevance, id, section) = refs[resource] - updateWordHTML(word, resource, section, id, relevance) - i = i + 1 - - print "Found %d associations in HTML pages" % (i) - -def analyzeAPITop(): - global wordsDict - global API - - try: - doc = loadAPI(API) - ret = analyzeAPI(doc) - print "Analyzed %d blocs" % (ret) - doc.freeDoc() - except: - print "Failed to parse and analyze %s" % (API) - print sys.exc_type, sys.exc_value - sys.exit(1) - - print "Indexed %d words" % (len(wordsDict)) - i = 0 - skipped = 0 - for word in wordsDict.keys(): - refs = wordsDict[word] - if refs is None: - skipped = skipped + 1 - continue - for (module, symbol) in refs.keys(): - updateWord(word, symbol, refs[(module, symbol)]) - i = i + 1 - - print "Found %d associations, skipped %d words" % (i, skipped) - -def usage(): - print "Usage index.py [--force] [--archive] [--archive-year year] [--archive-month month] [--API] [--docs]" - sys.exit(1) - -def main(): - try: - openMySQL() - except: - print "Failed to open the database" - print sys.exc_type, sys.exc_value - sys.exit(1) - - args = sys.argv[1:] - force = 0 - if args: - i = 0 - while i < len(args): - if args[i] == '--force': - force = 1 - elif args[i] == '--archive': - analyzeArchives(None, force) - elif args[i] == '--archive-year': - i = i + 1 - year = args[i] - months = ["January", "February", "March", "April", "May", - "June", "July", "August", "September", "October", - "November", "December"] - for month in months: - try: - str = "%s-%s" % (year, month) - T = time.strptime(str, "%Y-%B") - t = time.mktime(T) + 3600 * 24 * 10 - analyzeArchives(t, force) - except: - print "Failed to index month archive:" - print sys.exc_type, sys.exc_value - elif args[i] == '--archive-month': - i = i + 1 - month = args[i] - try: - T = time.strptime(month, "%Y-%B") - t = time.mktime(T) + 3600 * 24 * 10 - analyzeArchives(t, force) - except: - print "Failed to index month archive:" - print sys.exc_type, sys.exc_value - elif args[i] == '--API': - analyzeAPITop() - elif args[i] == '--docs': - analyzeHTMLTop() - else: - usage() - i = i + 1 - else: - usage() - -if __name__ == "__main__": - main() From patchwork Wed Apr 3 22:26:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Cole Robinson X-Patchwork-Id: 161717 Delivered-To: patch@linaro.org Received: by 2002:a02:c6d8:0:0:0:0:0 with SMTP id r24csp862368jan; Wed, 3 Apr 2019 15:27:15 -0700 (PDT) X-Google-Smtp-Source: APXvYqxKt/9YDFYL5PMETd4xaUavQGyWgw6s0eGYqiYvHsz5x7avh5QFOeF4WhKcYbeBeXCO63KI X-Received: by 2002:a37:aa58:: with SMTP id t85mr2263444qke.291.1554330435487; Wed, 03 Apr 2019 15:27:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1554330435; cv=none; d=google.com; s=arc-20160816; b=IyossRO3TpWuft5mwa5Gf9PFhOlVcl5z6/wlizSZLfq/PvjbluOIsdCByj6GdTkkjI z7DJ3UprYk1FNDqpfHSvGuZ2lVRoqmre4e/hNQBFjbJF6wobxXonK47FioPt30LPbAvH NNfikrqRHKrrz9moSfihdtF5twZF1f4uil2xVygTeGV19pq/c3LKrQNRQ5LJ1H2wuItL 5VeV7l2MyEMBNQfX3/Wn3xddArV0zjcEKDOOAK/rc7f/+JMU4D+oyFahnCa57PSvu32i sB1s2pAuNH8CtK9RMZHXouaAslkO1I/xjcZrCekpZtBopovDQRUy5QOd/HQJs1DH1XlR InSw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:sender:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :delivered-to; bh=xd+UrTuHid3cEc9d+3zRF5wSwBcBcs/lHM9o9pd7D6Y=; b=JTrMpbKpkzhGTJPYTZNNkK3FSJ1K9qrUoc67L4g5L+5fzgoOis6OTRyw6PdpVqMjnX F8Bmq7FEHbn/z1Sz4nG0Git+lR6oAmFpeLq5c3H1ZcO1DPbfjcx6DnvQcFz32kkCSS7A l2bgi94G3lsc9tp+kYRMs/iU7ksqH3NySt+f65h1NM3bU5MafwqlBAY6ZwFp+0JKyPaq 2zHedS/W91wej2A7VD6SjI3VUjvN3WtcljTEWONIlfUHXsFOuQMc1jVOrwoyV9ZLKKi3 KHOnE7NE033I8Y35Nr9ELiNNPTKkZ1V+7UgtBiOvoNz5WsyFreDKozZvsArXDarlnZ3L zNfA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of libvir-list-bounces@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id p44si3587525qvf.113.2019.04.03.15.27.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Apr 2019 15:27:15 -0700 (PDT) Received-SPF: pass (google.com: domain of libvir-list-bounces@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of libvir-list-bounces@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id F1E343082B5A; Wed, 3 Apr 2019 22:27:13 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CB7DB61082; Wed, 3 Apr 2019 22:27:13 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 8C89E38BE; Wed, 3 Apr 2019 22:27:13 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x33MR0Jx020298 for ; Wed, 3 Apr 2019 18:27:00 -0400 Received: by smtp.corp.redhat.com (Postfix) id 310AA10A33C5; Wed, 3 Apr 2019 22:27:00 +0000 (UTC) Delivered-To: libvirt-list@redhat.com Received: from worklaptop.redhat.com (ovpn-120-38.rdu2.redhat.com [10.10.120.38]) by smtp.corp.redhat.com (Postfix) with ESMTP id C155F10A33C1; Wed, 3 Apr 2019 22:26:59 +0000 (UTC) From: Cole Robinson To: libvirt-list@redhat.com Date: Wed, 3 Apr 2019 18:26:51 -0400 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH 3/3] docs: Remove search.php and all references X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Wed, 03 Apr 2019 22:27:14 +0000 (UTC) libvirt.org/search.php drops into some kind of screen which I guess is supposed to show a search bar with options, but presently for me renders as nothing but the following text: Search the documentation on Libvirt.org The search service indexes the libvirt APIs and documentation as well as the libvir-list@redhat.com mailing-list archives. To use it simply provide a set of keywords: The main page search bar now redirects to google, this page is broken, I say we just remove it and move on. Signed-off-by: Cole Robinson --- .gitignore | 1 - docs/Makefile.am | 22 +--- docs/devhelp/html.xsl | 4 - docs/search.php.code.in | 225 ---------------------------------------- docs/search.php.in | 16 --- 5 files changed, 2 insertions(+), 266 deletions(-) delete mode 100644 docs/search.php.code.in delete mode 100644 docs/search.php.in -- 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list Reviewed-by: Daniel P. Berrangé diff --git a/.gitignore b/.gitignore index c918ec8226..7f99e5db2d 100644 --- a/.gitignore +++ b/.gitignore @@ -66,7 +66,6 @@ /docs/libvirt-qemu-*.xml /docs/libvirt-refs.xml /docs/news.html.in -/docs/search.php /docs/todo.html.in /examples/admin/client_close /examples/admin/client_info diff --git a/docs/Makefile.am b/docs/Makefile.am index bd7bc1a431..ebdc734ddd 100644 --- a/docs/Makefile.am +++ b/docs/Makefile.am @@ -128,10 +128,6 @@ dot_html_in = \ $(notdir $(wildcard $(srcdir)/*.html.in)) dot_html = $(dot_html_in:%.html.in=%.html) -dot_php_in = $(notdir $(wildcard $(srcdir)/*.php.in)) -dot_php_code_in = $(dot_php_in:%.php.in=%.php.code.in) -dot_php = $(dot_php_in:%.php.in=%.php) - xml = \ libvirt-api.xml \ libvirt-refs.xml @@ -175,7 +171,7 @@ EXTRA_DIST= \ $(dot_html) $(dot_html_in) $(gif) $(apihtml) $(apipng) \ $(devhelphtml) $(devhelppng) $(devhelpcss) $(devhelpxsl) \ $(xml) $(qemu_xml) $(lxc_xml) $(admin_xml) $(fig) $(png) $(css) \ - $(logofiles) $(dot_php_in) $(dot_php_code_in) $(dot_php)\ + $(logofiles) \ $(internals_html_in) $(internals_html) $(fonts) \ aclperms.htmlinc \ hvsupport.pl \ @@ -192,7 +188,6 @@ MAINTAINERCLEANFILES = \ $(addprefix $(srcdir)/,$(apihtml)) \ $(addprefix $(srcdir)/,$(devhelphtml)) \ $(addprefix $(srcdir)/,$(internals_html)) \ - $(addprefix $(srcdir)/,$(dot_php)) \ $(srcdir)/hvsupport.html.in $(srcdir)/aclperms.htmlinc timestamp="$(shell if test -n "$$SOURCE_DATE_EPOCH"; \ @@ -209,8 +204,7 @@ qemu_api: $(srcdir)/libvirt-qemu-api.xml $(srcdir)/libvirt-qemu-refs.xml lxc_api: $(srcdir)/libvirt-lxc-api.xml $(srcdir)/libvirt-lxc-refs.xml admin_api: $(srcdir)/libvirt-admin-api.xml $(srcdir)/libvirt-admin-refs.xml -web: $(dot_html) $(internals_html) html/index.html devhelp/index.html \ - $(dot_php) +web: $(dot_html) $(internals_html) html/index.html devhelp/index.html hvsupport.html: $(srcdir)/hvsupport.html.in @@ -265,18 +259,6 @@ MAINTAINERCLEANFILES += \ $(AM_V_GEN)$(XMLLINT) --nonet --format $< > $(srcdir)/$@ \ || { rm $(srcdir)/$@ && exit 1; } -%.php.tmp: %.php.in site.xsl page.xsl - $(AM_V_GEN)$(XSLTPROC) --stringparam pagename $(@:.tmp=) \ - --stringparam timestamp $(timestamp) --nonet \ - $(top_srcdir)/docs/site.xsl $< > $@ \ - || { rm $@ && exit 1; } - -%.php: %.php.tmp %.php.code.in - $(AM_V_GEN)sed \ - -e '/<\/span>/r '"$(srcdir)/$@.code.in" \ - -e /php_placeholder/d < $@.tmp > $(srcdir)/$@ \ - || { rm $(srcdir)/$@ && exit 1; } - $(apihtml_generated): html/index.html html/index.html: libvirt-api.xml newapi.xsl page.xsl $(APIBUILD_STAMP) diff --git a/docs/devhelp/html.xsl b/docs/devhelp/html.xsl index eb10e362bf..9cdc049150 100644 --- a/docs/devhelp/html.xsl +++ b/docs/devhelp/html.xsl @@ -565,10 +565,6 @@ by a Linux instance. The library aim at providing long term stable C API initially for the Xen paravirtualization but should be able to integrate other virtualization mechanisms if needed.

-

If you get lost searching for some specific API use, try -the online search -engine hosted on libvirt.org -it indexes the project page, the APIs as well as the mailing-list archives.

diff --git a/docs/search.php.code.in b/docs/search.php.code.in deleted file mode 100644 index 01a6a64d28..0000000000 --- a/docs/search.php.code.in +++ /dev/null @@ -1,225 +0,0 @@ - - -
- - - -
- - $rb) ? -1 : 1; - } - if (($query) && (strlen($query) <= 50)) { - $link = mysql_connect ("localhost", "nobody"); - if (!$link) { - echo "

Could not connect to the database: ", mysql_error(); - } else { - mysql_select_db("libvir", $link); - $list = explode (" ", $query); - $results = array(); - $number = 0; - for ($number = 0;$number < count($list);$number++) { - - $word = $list[$number]; - if (($scope == 'any') || ($scope == 'API')) { - list($result, $j) = queryWord($word); - if ($j > 0) { - for ($i = 0; $i < $j; $i++) { - $relevance = mysql_result($result, $i, 0); - $name = mysql_result($result, $i, 1); - $type = mysql_result($result, $i, 2); - $module = mysql_result($result, $i, 3); - $desc = mysql_result($result, $i, 4); - if (array_key_exists($name, $results)) { - list($r,$t,$m,$d,$w,$u) = $results[$name]; - $results[$name] = array(($r + $relevance) * 2, - $t,$m,$d,$w,$u); - } else { - $id = $name; - $m = strtolower($module); - $url = "html/libvirt-$module.html#$id"; - $results[$name] = array($relevance,$type, - $module, $desc, $name, $url); - } - } - mysql_free_result($result); - } - } - if (($scope == 'any') || ($scope == 'DOCS')) { - list($result, $k) = queryHTMLWord($word); - if ($k > 0) { - for ($i = 0; $i < $k; $i++) { - $relevance = mysql_result($result, $i, 0); - $name = mysql_result($result, $i, 1); - $id = mysql_result($result, $i, 2); - $module = mysql_result($result, $i, 3); - $desc = mysql_result($result, $i, 4); - $url = $module; - if ($id != "") { - $url = $url + "#$id"; - } - $results["$name _html_ $number _ $i"] = - array($relevance, "XML docs", - $module, $desc, $name, $url); - } - mysql_free_result($result); - } - } - if (($scope == 'any') || ($scope == 'LISTS')) { - list($result, $j) = queryArchiveWord($word); - if ($j > 0) { - for ($i = 0; $i < $j; $i++) { - $relevance = mysql_result($result, $i, 0); - $name = mysql_result($result, $i, 1); - $type = mysql_result($result, $i, 2); - $url = mysql_result($result, $i, 3); - $desc = mysql_result($result, $i, 4); - if (array_key_exists($url, $results)) { - list($r,$t,$m,$d,$w,$u) = $results[$url]; - $results[$name] = array(($r + $relevance) * 2, - $t,$m,$d,$w,$u); - } else { - $id = $name; - $m = strtolower($module); - $u = str_replace( - "http://www.redhat.com/archives/libvir-list/", "", $url); - $results[$url] = array($relevance,$type, - $u, $desc, $name, $url); - } - } - mysql_free_result($result); - } - } - } - if ((count($results) == 0) && (count($list) == 1)) { - $word = $list[0]; - if (($scope == 'any') || ($scope == 'XMLAPI')) { - list($result, $j) = queryWord("vir$word"); - if ($j > 0) { - for ($i = 0; $i < $j; $i++) { - $relevance = mysql_result($result, $i, 0); - $name = mysql_result($result, $i, 1); - $type = mysql_result($result, $i, 2); - $module = mysql_result($result, $i, 3); - $desc = mysql_result($result, $i, 4); - if (array_key_exists($name, $results)) { - list($r,$t,$m,$d,$w,$u) = $results[$name]; - $results[$name] = array(($r + $relevance) * 2, - $t,$m,$d,$w,$u); - } else { - $id = $name; - $m = strtolower($module); - $url = "html/libvirt-$module.html#$id"; - $results[$name] = array($relevance,$type, - $module, $desc, $name, $url); - } - } - mysql_free_result($result); - } - } - } - mysql_close($link); - $nb = count($results); - echo "

Found $nb results for query $querystr

\n"; - usort($results, "resSort"); - - if ($nb > 0) { - printf("\n"); - printf("\n"); - $i = 0; - while (list ($name, $val) = each ($results)) { - list($r,$t,$m,$d,$s,$u) = $val; - $m = str_replace("<", "<", $m); - $s = str_replace("<", "<", $s); - $d = str_replace("<", "<", $d); - echo ""; - $i = $i + 1; - if ($i > 75) - break; - } - printf("
QualitySymbolTypemoduleDescription
$r$s$t$m$d
\n"); - } - } - } -?> diff --git a/docs/search.php.in b/docs/search.php.in deleted file mode 100644 index 5de4fcee66..0000000000 --- a/docs/search.php.in +++ /dev/null @@ -1,16 +0,0 @@ - - - - -

Search the documentation on Libvirt.org

- -

- The search service indexes the libvirt APIs and documentation as - well as the libvir-list@redhat.com mailing-list archives. To use - it simply provide a set of keywords: -

- - - - -