[PATCHv2,00/32] OEQA Framework Refactor & Improvements

Message ID cover.1481142664.git.anibal.limon@linux.intel.com
State New
Headers show

Pull-request

git://git.yoctoproject.org/poky-contrib alimon/oeqa_sdk_migration

Message

Aníbal Limón Dec. 7, 2016, 8:32 p.m.
This patchset is related to OEQA Framework for details read the RFC send to the
Openembedded architecture ML.

http://lists.openembedded.org/pipermail/openembedded-architecture/2016-December/000351.html

This v2 fixes sdk extensible test case sdk_update.

The following changes since commit 9e63f81c78e284c9b325fe04a1b59e61c7ad8a1a:

  bitbake: ast: remove BBVERSIONS support (2016-11-30 15:48:10 +0000)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib alimon/oeqa_sdk_migration
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=alimon/oeqa_sdk_migration

Aníbal Limón (28):
  oeqa/core: Add base OEQA framework
  oeqa/core: Add loader, context and decorator modules
  oeqa/core/decorator: Add support for OETestDepends
  oeqa/core/decorator: Add support for OETestDataDepends and
    skipIfDataVar
  scripts/oe-test: Add new oe-test script
  oeqa/core/context: Add support of OETestContextExecutor
  oeqa/core/cases: Add example test cases
  oeqa/core: Add README
  oe/data: Add export2json function
  classes/rootfs-postcommands: Add write_image_test_data
  classes/populate_sdk_base: Add write_sdk_test_data to postprocess
  oeqa/core: Change name of d to td
  oeqa/utils/path: Add remove_safe function
  oeqa: Move common files to oeqa/files instead of runtime only
  oeqa/sdk: Move test cases inside cases directory
  oeqa/{runtime,sdk}/files: Move testsdkmakefile from runtime to sdk
    module
  oeqa/sdk: Add case and context modules for the SDK component
  classes/testsdk: Migrates testsdk.bbclass to use new OESDKTestContext
  oeqa/utils: Move targetbuild to buildproject module
  oeqa/utils: {Target,SDK,}BuildProject remove dependency of bb
  oeqa/sdk/cases: Migrate tests to the new OEQA framework
  classes/testsdk: Remove the need of TEST_LOG_DIR variable
  oeqa/sdkext: Move test cases inside cases directory
  oeqa/sdkext: Adds case and context modules.
  classes/testsdk: Migrate to use the new OESDKExtTestContext
  oeqa/sdkext/cases: Migrate test case to new OEQA framework
  oeqa/runtime: Fix TargetBuildProject instances
  oeqa: Fix files handling on runtime tests.

Mariano Lopez (4):
  oeqa/core: Add utils module for OEQA framework
  oeqa/core/decorator: Add support for OETestID and OETestTag
  oeqa/core/decorator: Add support for OETimeout decorator
  oeqa/core: Add tests for the OEQA framework

 meta/classes/populate_sdk_base.bbclass             |   9 +-
 meta/classes/rootfs-postcommands.bbclass           |  18 ++
 meta/classes/testexport.bbclass                    |   5 +
 meta/classes/testimage.bbclass                     |   4 +
 meta/classes/testsdk.bbclass                       | 162 ++++++++------
 meta/lib/oe/data.py                                |  28 +++
 meta/lib/oeqa/core/README                          |  38 ++++
 meta/lib/oeqa/core/__init__.py                     |   0
 meta/lib/oeqa/core/case.py                         |  46 ++++
 meta/lib/oeqa/core/cases/__init__.py               |   0
 meta/lib/oeqa/core/cases/example/data.json         |   1 +
 meta/lib/oeqa/core/cases/example/test_basic.py     |  20 ++
 meta/lib/oeqa/core/context.py                      | 225 ++++++++++++++++++++
 meta/lib/oeqa/core/decorator/__init__.py           |  71 +++++++
 meta/lib/oeqa/core/decorator/data.py               |  36 ++++
 meta/lib/oeqa/core/decorator/depends.py            |  94 +++++++++
 meta/lib/oeqa/core/decorator/oeid.py               |  23 ++
 meta/lib/oeqa/core/decorator/oetag.py              |  24 +++
 meta/lib/oeqa/core/decorator/oetimeout.py          |  25 +++
 meta/lib/oeqa/core/exception.py                    |  14 ++
 meta/lib/oeqa/core/loader.py                       | 235 +++++++++++++++++++++
 meta/lib/oeqa/core/runner.py                       |  76 +++++++
 meta/lib/oeqa/core/tests/__init__.py               |   0
 meta/lib/oeqa/core/tests/cases/data.py             |  20 ++
 meta/lib/oeqa/core/tests/cases/depends.py          |  38 ++++
 .../oeqa/core/tests/cases/loader/invalid/oeid.py   |  15 ++
 .../oeqa/core/tests/cases/loader/valid/another.py  |   9 +
 meta/lib/oeqa/core/tests/cases/oeid.py             |  18 ++
 meta/lib/oeqa/core/tests/cases/oetag.py            |  18 ++
 meta/lib/oeqa/core/tests/cases/timeout.py          |  18 ++
 meta/lib/oeqa/core/tests/common.py                 |  35 +++
 meta/lib/oeqa/core/tests/test_data.py              |  51 +++++
 meta/lib/oeqa/core/tests/test_decorators.py        | 135 ++++++++++++
 meta/lib/oeqa/core/tests/test_loader.py            |  86 ++++++++
 meta/lib/oeqa/core/tests/test_runner.py            |  38 ++++
 meta/lib/oeqa/core/utils/__init__.py               |   0
 meta/lib/oeqa/core/utils/misc.py                   |  37 ++++
 meta/lib/oeqa/core/utils/path.py                   |  19 ++
 meta/lib/oeqa/core/utils/test.py                   |  71 +++++++
 meta/lib/oeqa/{runtime => }/files/test.c           |   0
 meta/lib/oeqa/{runtime => }/files/test.cpp         |   0
 meta/lib/oeqa/{runtime => }/files/test.pl          |   0
 meta/lib/oeqa/{runtime => }/files/test.py          |   0
 meta/lib/oeqa/oetest.py                            |  92 +-------
 meta/lib/oeqa/runtime/buildcvs.py                  |   9 +-
 meta/lib/oeqa/runtime/buildgalculator.py           |   8 +-
 meta/lib/oeqa/runtime/buildiptables.py             |   8 +-
 meta/lib/oeqa/runtime/gcc.py                       |   4 +-
 meta/lib/oeqa/runtime/perl.py                      |   2 +-
 meta/lib/oeqa/runtime/python.py                    |   2 +-
 meta/lib/oeqa/runtime/utils/__init__.py            |   0
 meta/lib/oeqa/runtime/utils/targetbuildproject.py  |  33 +++
 meta/lib/oeqa/sdk/__init__.py                      |   3 -
 meta/lib/oeqa/sdk/case.py                          |  12 ++
 meta/lib/oeqa/sdk/{ => cases}/buildcvs.py          |  15 +-
 meta/lib/oeqa/sdk/{ => cases}/buildgalculator.py   |  28 ++-
 meta/lib/oeqa/sdk/{ => cases}/buildiptables.py     |  16 +-
 meta/lib/oeqa/sdk/cases/gcc.py                     |  43 ++++
 meta/lib/oeqa/sdk/cases/perl.py                    |  27 +++
 meta/lib/oeqa/sdk/{ => cases}/python.py            |  25 ++-
 meta/lib/oeqa/sdk/context.py                       | 133 ++++++++++++
 .../oeqa/{runtime => sdk}/files/testsdkmakefile    |   0
 meta/lib/oeqa/sdk/gcc.py                           |  36 ----
 meta/lib/oeqa/sdk/perl.py                          |  28 ---
 meta/lib/oeqa/sdk/utils/__init__.py                |   0
 meta/lib/oeqa/sdk/utils/sdkbuildproject.py         |  45 ++++
 meta/lib/oeqa/sdkext/__init__.py                   |   3 -
 meta/lib/oeqa/sdkext/case.py                       |  21 ++
 meta/lib/oeqa/sdkext/{ => cases}/devtool.py        |  49 +++--
 meta/lib/oeqa/sdkext/{ => cases}/sdk_update.py     |  17 +-
 meta/lib/oeqa/sdkext/context.py                    |  21 ++
 meta/lib/oeqa/utils/buildproject.py                |  52 +++++
 meta/lib/oeqa/utils/targetbuild.py                 | 135 ------------
 scripts/oe-test                                    |  98 +++++++++
 74 files changed, 2284 insertions(+), 443 deletions(-)
 create mode 100644 meta/lib/oeqa/core/README
 create mode 100644 meta/lib/oeqa/core/__init__.py
 create mode 100644 meta/lib/oeqa/core/case.py
 create mode 100644 meta/lib/oeqa/core/cases/__init__.py
 create mode 100644 meta/lib/oeqa/core/cases/example/data.json
 create mode 100644 meta/lib/oeqa/core/cases/example/test_basic.py
 create mode 100644 meta/lib/oeqa/core/context.py
 create mode 100644 meta/lib/oeqa/core/decorator/__init__.py
 create mode 100644 meta/lib/oeqa/core/decorator/data.py
 create mode 100644 meta/lib/oeqa/core/decorator/depends.py
 create mode 100644 meta/lib/oeqa/core/decorator/oeid.py
 create mode 100644 meta/lib/oeqa/core/decorator/oetag.py
 create mode 100644 meta/lib/oeqa/core/decorator/oetimeout.py
 create mode 100644 meta/lib/oeqa/core/exception.py
 create mode 100644 meta/lib/oeqa/core/loader.py
 create mode 100644 meta/lib/oeqa/core/runner.py
 create mode 100644 meta/lib/oeqa/core/tests/__init__.py
 create mode 100644 meta/lib/oeqa/core/tests/cases/data.py
 create mode 100644 meta/lib/oeqa/core/tests/cases/depends.py
 create mode 100644 meta/lib/oeqa/core/tests/cases/loader/invalid/oeid.py
 create mode 100644 meta/lib/oeqa/core/tests/cases/loader/valid/another.py
 create mode 100644 meta/lib/oeqa/core/tests/cases/oeid.py
 create mode 100644 meta/lib/oeqa/core/tests/cases/oetag.py
 create mode 100644 meta/lib/oeqa/core/tests/cases/timeout.py
 create mode 100644 meta/lib/oeqa/core/tests/common.py
 create mode 100755 meta/lib/oeqa/core/tests/test_data.py
 create mode 100755 meta/lib/oeqa/core/tests/test_decorators.py
 create mode 100755 meta/lib/oeqa/core/tests/test_loader.py
 create mode 100755 meta/lib/oeqa/core/tests/test_runner.py
 create mode 100644 meta/lib/oeqa/core/utils/__init__.py
 create mode 100644 meta/lib/oeqa/core/utils/misc.py
 create mode 100644 meta/lib/oeqa/core/utils/path.py
 create mode 100644 meta/lib/oeqa/core/utils/test.py
 rename meta/lib/oeqa/{runtime => }/files/test.c (100%)
 rename meta/lib/oeqa/{runtime => }/files/test.cpp (100%)
 rename meta/lib/oeqa/{runtime => }/files/test.pl (100%)
 rename meta/lib/oeqa/{runtime => }/files/test.py (100%)
 create mode 100644 meta/lib/oeqa/runtime/utils/__init__.py
 create mode 100644 meta/lib/oeqa/runtime/utils/targetbuildproject.py
 create mode 100644 meta/lib/oeqa/sdk/case.py
 rename meta/lib/oeqa/sdk/{ => cases}/buildcvs.py (59%)
 rename meta/lib/oeqa/sdk/{ => cases}/buildgalculator.py (45%)
 rename meta/lib/oeqa/sdk/{ => cases}/buildiptables.py (58%)
 create mode 100644 meta/lib/oeqa/sdk/cases/gcc.py
 create mode 100644 meta/lib/oeqa/sdk/cases/perl.py
 rename meta/lib/oeqa/sdk/{ => cases}/python.py (46%)
 create mode 100644 meta/lib/oeqa/sdk/context.py
 rename meta/lib/oeqa/{runtime => sdk}/files/testsdkmakefile (100%)
 delete mode 100644 meta/lib/oeqa/sdk/gcc.py
 delete mode 100644 meta/lib/oeqa/sdk/perl.py
 create mode 100644 meta/lib/oeqa/sdk/utils/__init__.py
 create mode 100644 meta/lib/oeqa/sdk/utils/sdkbuildproject.py
 create mode 100644 meta/lib/oeqa/sdkext/case.py
 rename meta/lib/oeqa/sdkext/{ => cases}/devtool.py (73%)
 rename meta/lib/oeqa/sdkext/{ => cases}/sdk_update.py (63%)
 create mode 100644 meta/lib/oeqa/sdkext/context.py
 create mode 100644 meta/lib/oeqa/utils/buildproject.py
 delete mode 100644 meta/lib/oeqa/utils/targetbuild.py
 create mode 100755 scripts/oe-test

Comments

Ross Burton Dec. 8, 2016, 2:41 p.m. | #1
On 7 December 2016 at 20:32, Aníbal Limón <anibal.limon@linux.intel.com>
wrote:

> The d variable references the test data into a test context, so

> makes a more sense to call it: td (test data).

>


Can you squash this into the commits where they are introduced (don't need
to see your working, etc).

Ross
-- 
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Ross Burton Dec. 8, 2016, 2:41 p.m. | #2
Can you squash this into the commit that added core/utils/path.

Ross

On 7 December 2016 at 20:32, Aníbal Limón <anibal.limon@linux.intel.com>
wrote:

> Checks if path exists before try to remove of avoid exception.

>

> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>

> ---

>  meta/lib/oeqa/core/utils/path.py | 5 +++++

>  1 file changed, 5 insertions(+)

>

> diff --git a/meta/lib/oeqa/core/utils/path.py b/meta/lib/oeqa/core/utils/

> path.py

> index cb06523..a21caad 100644

> --- a/meta/lib/oeqa/core/utils/path.py

> +++ b/meta/lib/oeqa/core/utils/path.py

> @@ -12,3 +12,8 @@ def findFile(file_name, directory):

>          if file_name in f:

>              return os.path.join(r, file_name)

>      return None

> +

> +def remove_safe(path):

> +    if os.path.exists(path):

> +        os.remove(path)

> +

> --

> 2.1.4

>

> --

> _______________________________________________

> Openembedded-core mailing list

> Openembedded-core@lists.openembedded.org

> http://lists.openembedded.org/mailman/listinfo/openembedded-core

>
-- 
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Aníbal Limón Dec. 8, 2016, 4:06 p.m. | #3
Sure, sending v3.

	alimon

On 12/08/2016 08:41 AM, Burton, Ross wrote:
> On 7 December 2016 at 20:32, Aníbal Limón <anibal.limon@linux.intel.com>

> wrote:

> 

>> The d variable references the test data into a test context, so

>> makes a more sense to call it: td (test data).

>>

> 

> Can you squash this into the commits where they are introduced (don't need

> to see your working, etc).

> 

> Ross

>
-- 
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core