diff mbox series

[net,1/1] tc-testing: Update police test cases

Message ID 20210708080006.3687598-1-roid@nvidia.com
State New
Headers show
Series [net,1/1] tc-testing: Update police test cases | expand

Commit Message

Roi Dayan July 8, 2021, 8 a.m. UTC 
Update to match fixed output.

Signed-off-by: Roi Dayan <roid@nvidia.com>
---

Notes:
    Hi,
    
    This is related to commit that was merged
    
    55abdcf20a57 police: Add support for json output
    
    and also submitted another small fix commit titled
    "police: Small corrections for the output"
    
    Thanks,
    Roi

 .../tc-testing/tc-tests/actions/police.json   | 62 +++++++++----------
 1 file changed, 31 insertions(+), 31 deletions(-)

Comments

Jamal Hadi Salim July 8, 2021, 11:17 a.m. UTC | #1 
On 2021-07-08 4:00 a.m., Roi Dayan wrote:
> Update to match fixed output.
> 
> Signed-off-by: Roi Dayan <roid@nvidia.com>
> ---
> 
> Notes:
>      Hi,
>      
>      This is related to commit that was merged
>      
>      55abdcf20a57 police: Add support for json output
>      
>      and also submitted another small fix commit titled
>      "police: Small corrections for the output"
>      
>      Thanks,
>      Roi
> 
>   .../tc-testing/tc-tests/actions/police.json   | 62 +++++++++----------
>   1 file changed, 31 insertions(+), 31 deletions(-)
> 
> diff --git a/tools/testing/selftests/tc-testing/tc-tests/actions/police.json b/tools/testing/selftests/tc-testing/tc-tests/actions/police.json
> index 8e45792703ed..c9623c7afbd1 100644
> --- a/tools/testing/selftests/tc-testing/tc-tests/actions/police.json
> +++ b/tools/testing/selftests/tc-testing/tc-tests/actions/police.json
> @@ -17,7 +17,7 @@
>           "cmdUnderTest": "$TC actions add action police rate 1kbit burst 10k index 1",
>           "expExitCode": "0",
>           "verifyCmd": "$TC actions ls action police",
> -        "matchPattern": "action order [0-9]*:  police 0x1 rate 1Kbit burst 10Kb",
> +        "matchPattern": "action order [0-9]*: police.*index 1 rate 1Kbit burst 10Kb",
>           "matchCount": "1",
>           "teardown": [
>               "$TC actions flush action police"

Does the old output continue to work here?

cheers,
jamal
Jamal Hadi Salim July 8, 2021, 1:05 p.m. UTC | #2 
On 2021-07-08 8:11 a.m., Roi Dayan wrote:

[..]
> 
> no. old output doesn't have the string "index" and also output of index
> is in hex.
> 
> it is possible to make the old version work by allowing without index
> and looking for either the unsigned number or hex number/
> 
> but why do we need the old output to work? could use the "old" version
> of the test.

I think that would work if you assume this is only going to run
on the same kernel. But:
In this case because output json, which provides a formally parseable
output, then very likely someone's scripts are dependent on the old
output out there. So things have to be backward/forward compatible.
The new output does look better.
Maybe one approach is to have multiple matchPattern in the tests?
Davide?
We will have to deal with support issues when someone says their
script is broken.


cheers,
jamal
Davide Caratti July 8, 2021, 6:17 p.m. UTC | #3 
On Thu, Jul 08, 2021 at 09:05:03AM -0400, Jamal Hadi Salim wrote:
> On 2021-07-08 8:11 a.m., Roi Dayan wrote:
> 
> [..]
> > 
> > no. old output doesn't have the string "index" and also output of index
> > is in hex.
> > 
> > it is possible to make the old version work by allowing without index
> > and looking for either the unsigned number or hex number/
> > 
> > but why do we need the old output to work? could use the "old" version
> > of the test.
> 
> I think that would work if you assume this is only going to run
> on the same kernel. But:

in my perspective, that's already an issue (though a small one), because
somebody needs to ensure that a kernel is tested against a specific version
of iproute2. I say "small", because most probably the test will be waived
or skipped until all the changes (in iproute and kselftests) propagate.

> In this case because output json, which provides a formally parseable
> output, then very likely someone's scripts are dependent on the old
> output out there. So things have to be backward/forward compatible.
> The new output does look better.

the JSON output is something newly introduced  by this  commit, and I 
agree  it's good / sane to see 'index' and decimal printout in there,
like in other TC actions.

> Maybe one approach is to have multiple matchPattern in the tests?
> Davide?

sure, that's a possibility, but then:

> We will have to deal with support issues when someone says their
> script is broken.

that's probably something we need to care of. TC police is there since a
lot of time, so there might be users expecting no "index" and hex
printout in the human-readable format. If you use the old format
string with PRINT_FP, and the new one with PRINT_JSON, you should be
able to run tools/testing/selftests/net/forwarding/* and
tools/testing/selftests/tc-testing with the same iproute program,
with no "spoecial handling" for act_police.

WDYT?
Roi Dayan July 12, 2021, 12:29 p.m. UTC | #4 
On 2021-07-08 4:05 PM, Jamal Hadi Salim wrote:
> On 2021-07-08 8:11 a.m., Roi Dayan wrote:

> 

> [..]

>>

>> no. old output doesn't have the string "index" and also output of index

>> is in hex.

>>

>> it is possible to make the old version work by allowing without index

>> and looking for either the unsigned number or hex number/

>>

>> but why do we need the old output to work? could use the "old" version

>> of the test.

> 

> I think that would work if you assume this is only going to run

> on the same kernel. But:

> In this case because output json, which provides a formally parseable

> output, then very likely someone's scripts are dependent on the old

> output out there. So things have to be backward/forward compatible.

> The new output does look better.

> Maybe one approach is to have multiple matchPattern in the tests?

> Davide?

> We will have to deal with support issues when someone says their

> script is broken.

> 

> 

> cheers,

> jamal



this patch should be ignored now.
see "police: Fix normal output back to what it was"
diff mbox series

Patch

diff --git a/tools/testing/selftests/tc-testing/tc-tests/actions/police.json b/tools/testing/selftests/tc-testing/tc-tests/actions/police.json
index 8e45792703ed..c9623c7afbd1 100644
--- a/tools/testing/selftests/tc-testing/tc-tests/actions/police.json
+++ b/tools/testing/selftests/tc-testing/tc-tests/actions/police.json
@@ -17,7 +17,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 1kbit burst 10k index 1",
         "expExitCode": "0",
         "verifyCmd": "$TC actions ls action police",
-        "matchPattern": "action order [0-9]*:  police 0x1 rate 1Kbit burst 10Kb",
+        "matchPattern": "action order [0-9]*: police.*index 1 rate 1Kbit burst 10Kb",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -42,7 +42,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 8kbit burst 24k index 9",
         "expExitCode": "255",
         "verifyCmd": "$TC actions ls action police",
-        "matchPattern": "action order [0-9]*:  police 0x9",
+        "matchPattern": "action order [0-9]*: police.*index 9",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -66,7 +66,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 90kbit burst 10k mtu 1k index 98",
         "expExitCode": "0",
         "verifyCmd": "$TC actions get action police index 98",
-        "matchPattern": "action order [0-9]*:  police 0x62 rate 90Kbit burst 10Kb mtu 1Kb",
+        "matchPattern": "action order [0-9]*: police.*index 98 rate 90Kbit burst 10Kb mtu 1Kb",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -90,7 +90,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 90kbit burst 10k mtu 2kb peakrate 100kbit index 3",
         "expExitCode": "0",
         "verifyCmd": "$TC actions ls action police",
-        "matchPattern": "action order [0-9]*:  police 0x3 rate 90Kbit burst 10Kb mtu 2Kb peakrate 100Kbit",
+        "matchPattern": "action order [0-9]*: police.*index 3 rate 90Kbit burst 10Kb mtu 2Kb peakrate 100Kbit",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -114,7 +114,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 5kbit burst 6kb peakrate 10kbit index 9",
         "expExitCode": "255",
         "verifyCmd": "$TC actions ls action police",
-        "matchPattern": "action order [0-9]*:  police 0x9 rate 5Kb burst 10Kb",
+        "matchPattern": "action order [0-9]*: police.*index 9 rate 5Kb burst 10Kb",
         "matchCount": "0",
         "teardown": [
             "$TC actions flush action police"
@@ -138,7 +138,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 1mbit burst 100k overhead 64 index 64",
         "expExitCode": "0",
         "verifyCmd": "$TC actions get action police index 64",
-        "matchPattern": "action order [0-9]*:  police 0x40 rate 1Mbit burst 100Kb mtu 2Kb action reclassify overhead 64b",
+        "matchPattern": "action order [0-9]*: police.*index 64 rate 1Mbit burst 100Kb mtu 2Kb action reclassify overhead 64b",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -162,7 +162,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 2mbit burst 200k linklayer ethernet index 8",
         "expExitCode": "0",
         "verifyCmd": "$TC actions show action police",
-        "matchPattern": "action order [0-9]*:  police 0x8 rate 2Mbit burst 200Kb mtu 2Kb action reclassify overhead 0b",
+        "matchPattern": "action order [0-9]*: police.*index 8 rate 2Mbit burst 200Kb mtu 2Kb action reclassify overhead 0b",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -186,7 +186,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 2mbit burst 200k linklayer atm index 8",
         "expExitCode": "0",
         "verifyCmd": "$TC actions show action police",
-        "matchPattern": "action order [0-9]*:  police 0x8 rate 2Mbit burst 200Kb mtu 2Kb action reclassify overhead 0b linklayer atm",
+        "matchPattern": "action order [0-9]*: police.*index 8 rate 2Mbit burst 200Kb mtu 2Kb action reclassify overhead 0b linklayer atm",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -210,7 +210,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 3mbit burst 250k conform-exceed continue/drop index 1",
         "expExitCode": "0",
         "verifyCmd": "$TC actions get action police index 1",
-        "matchPattern": "action order [0-9]*:  police 0x1 rate 3Mbit burst 250Kb mtu 2Kb action continue/drop",
+        "matchPattern": "action order [0-9]*: police.*index 1 rate 3Mbit burst 250Kb mtu 2Kb action continue/drop",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -234,7 +234,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 3mbit burst 250k conform-exceed pass/reclassify index 4",
         "expExitCode": "0",
         "verifyCmd": "$TC actions ls action police",
-        "matchPattern": "action order [0-9]*:  police 0x4 rate 3Mbit burst 250Kb mtu 2Kb action pass/reclassify",
+        "matchPattern": "action order [0-9]*: police.*index 4 rate 3Mbit burst 250Kb mtu 2Kb action pass/reclassify",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -258,7 +258,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 3mbit burst 250k conform-exceed pass/pipe index 5",
         "expExitCode": "0",
         "verifyCmd": "$TC actions ls action police",
-        "matchPattern": "action order [0-9]*:  police 0x5 rate 3Mbit burst 250Kb mtu 2Kb action pass/pipe",
+        "matchPattern": "action order [0-9]*: police.*index 5 rate 3Mbit burst 250Kb mtu 2Kb action pass/pipe",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -282,7 +282,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 3tb burst 250k conform-exceed pass/pipe index 5",
         "expExitCode": "255",
         "verifyCmd": "$TC actions ls action police",
-        "matchPattern": "action order [0-9]*:  police 0x5 rate 3Tb burst 250Kb mtu 2Kb action pass/pipe",
+        "matchPattern": "action order [0-9]*: police.*index 5 rate 3Tb burst 250Kb mtu 2Kb action pass/pipe",
         "matchCount": "0",
         "teardown": [
             "$TC actions flush action police"
@@ -306,7 +306,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 3kbit burst 250P conform-exceed pass/pipe index 5",
         "expExitCode": "255",
         "verifyCmd": "$TC actions ls action police",
-        "matchPattern": "action order [0-9]*:  police 0x5 rate 3Kbit burst 250Pb mtu 2Kb action pass/pipe",
+        "matchPattern": "action order [0-9]*: police.*index 5 rate 3Kbit burst 250Pb mtu 2Kb action pass/pipe",
         "matchCount": "0",
         "teardown": [
             "$TC actions flush action police"
@@ -330,7 +330,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 3mbit burst 250k conform-exceed 0/3 index 1",
         "expExitCode": "0",
         "verifyCmd": "$TC actions get action police index 1",
-        "matchPattern": "action order [0-9]*:  police 0x1 rate 3Mbit burst 250Kb mtu 2Kb action pass/pipe",
+        "matchPattern": "action order [0-9]*: police.*index 1 rate 3Mbit burst 250Kb mtu 2Kb action pass/pipe",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -354,7 +354,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 3mbit burst 250k conform-exceed 10/drop index 1",
         "expExitCode": "255",
         "verifyCmd": "$TC actions ls action police",
-        "matchPattern": "action order [0-9]*:  police 0x1 rate 3Mbit burst 250Kb mtu 2Kb action ",
+        "matchPattern": "action order [0-9]*: police.*index 1 rate 3Mbit burst 250Kb mtu 2Kb action ",
         "matchCount": "0",
         "teardown": [
             "$TC actions flush action police"
@@ -378,7 +378,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 90kbit burst 10k mtu 2kb peakrate 100T index 1",
         "expExitCode": "255",
         "verifyCmd": "$TC actions ls action police",
-        "matchPattern": "action order [0-9]*:  police 0x1 rate 90Kbit burst 10Kb mtu 2Kb peakrate 100Tbit",
+        "matchPattern": "action order [0-9]*: police.*index 1 rate 90Kbit burst 10Kb mtu 2Kb peakrate 100Tbit",
         "matchCount": "0",
         "teardown": [
             "$TC actions flush action police"
@@ -402,7 +402,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 10kbit burst 10k mtu 2Pbit index 1",
         "expExitCode": "255",
         "verifyCmd": "$TC actions ls action police",
-        "matchPattern": "action order [0-9]*:  police 0x1 rate 10Kbit burst 1Kb mtu 2Pb",
+        "matchPattern": "action order [0-9]*: police.*index 1 rate 10Kbit burst 1Kb mtu 2Pb",
         "matchCount": "0",
         "teardown": [
             "$TC actions flush action police"
@@ -426,7 +426,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 10mbit burst 10k index 1 cookie a1b1c1d1e1f12233bb",
         "expExitCode": "0",
         "verifyCmd": "$TC actions get action police index 1",
-        "matchPattern": "action order [0-9]*:  police 0x1 rate 10Mbit burst 10Kb mtu 2Kb.*cookie a1b1c1d1e1f12233bb",
+        "matchPattern": "action order [0-9]*: police.*index 1 rate 10Mbit burst 10Kb mtu 2Kb.*cookie a1b1c1d1e1f12233bb",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -450,7 +450,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 10mbit burst 10k index 4294967295",
         "expExitCode": "0",
         "verifyCmd": "$TC actions get action police index 4294967295",
-        "matchPattern": "action order [0-9]*:  police 0xffffffff rate 10Mbit burst 10Kb mtu 2Kb",
+        "matchPattern": "action order [0-9]*: police.*index 4294967295 rate 10Mbit burst 10Kb mtu 2Kb",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -475,7 +475,7 @@ 
         "cmdUnderTest": "$TC actions delete action police index 12",
         "expExitCode": "0",
         "verifyCmd": "$TC actions ls action police",
-        "matchPattern": "action order [0-9]*:  police 0xc rate 5Mb burst 2Mb",
+        "matchPattern": "action order [0-9]*: police.*index 12 rate 5Mb burst 2Mb",
         "matchCount": "0",
         "teardown": [
             "$TC actions flush action police"
@@ -507,7 +507,7 @@ 
         "cmdUnderTest": "$TC actions get action police index 4",
         "expExitCode": "0",
         "verifyCmd": "$TC actions get action police index 4",
-        "matchPattern": "action order [0-9]*:  police 0x4 rate 4Mbit burst 400Kb",
+        "matchPattern": "action order [0-9]*: police.*index 4 rate 4Mbit burst 400Kb",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -564,7 +564,7 @@ 
         "cmdUnderTest": "$TC actions list action police",
         "expExitCode": "0",
         "verifyCmd": "$TC actions ls action police",
-        "matchPattern": "action order [0-9]*:  police 0x[1-8] rate [1-8]Mbit burst [1-8]00Kb",
+        "matchPattern": "action order [0-9]*: police\\s*index [1-8] rate [1-8]Mbit burst [1-8]00Kb",
         "matchCount": "8",
         "teardown": [
             "$TC actions flush action police"
@@ -614,7 +614,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 7mbit burst 1m continue index 1",
         "expExitCode": "0",
         "verifyCmd": "$TC actions get action police index 1",
-        "matchPattern": "action order [0-9]*:  police 0x1 rate 7Mbit burst 1024Kb mtu 2Kb action continue",
+        "matchPattern": "action order [0-9]*: police.*index 1 rate 7Mbit burst 1024Kb mtu 2Kb action continue",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -638,7 +638,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 7mbit burst 1m drop index 1",
         "expExitCode": "0",
         "verifyCmd": "$TC actions ls action police",
-        "matchPattern": "action order [0-9]*:  police 0x1 rate 7Mbit burst 1024Kb mtu 2Kb action drop",
+        "matchPattern": "action order [0-9]*: police.*index 1 rate 7Mbit burst 1024Kb mtu 2Kb action drop",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -662,7 +662,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 7mbit burst 1m ok index 1",
         "expExitCode": "0",
         "verifyCmd": "$TC actions ls action police",
-        "matchPattern": "action order [0-9]*:  police 0x1 rate 7Mbit burst 1024Kb mtu 2Kb action pass",
+        "matchPattern": "action order [0-9]*: police.*index 1 rate 7Mbit burst 1024Kb mtu 2Kb action pass",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -686,7 +686,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 7mbit burst 1m reclassify index 1",
         "expExitCode": "0",
         "verifyCmd": "$TC actions get action police index 1",
-        "matchPattern": "action order [0-9]*:  police 0x1 rate 7Mbit burst 1024Kb mtu 2Kb action reclassify",
+        "matchPattern": "action order [0-9]*: police.*index 1 rate 7Mbit burst 1024Kb mtu 2Kb action reclassify",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -710,7 +710,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 7mbit burst 1m pipe index 1",
         "expExitCode": "0",
         "verifyCmd": "$TC actions ls action police",
-        "matchPattern": "action order [0-9]*:  police 0x1 rate 7Mbit burst 1024Kb mtu 2Kb action pipe",
+        "matchPattern": "action order [0-9]*: police.*index 1 rate 7Mbit burst 1024Kb mtu 2Kb action pipe",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -734,7 +734,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 1mbit burst 1k conform-exceed pass / goto chain 42",
         "expExitCode": "255",
         "verifyCmd": "$TC actions ls action police",
-        "matchPattern": "action order [0-9]*:  police 0x1 rate 1Mbit burst 1Kb mtu 2Kb action pass/goto chain 42",
+        "matchPattern": "action order [0-9]*: police.*index 1 rate 1Mbit burst 1Kb mtu 2Kb action pass/goto chain 42",
         "matchCount": "0",
         "teardown": [
             "$TC actions flush action police"
@@ -759,7 +759,7 @@ 
         "cmdUnderTest": "$TC actions replace action police rate 3mbit burst 250k goto chain 42 index 90 cookie c1a0c1a0",
         "expExitCode": "255",
         "verifyCmd": "$TC actions get action police index 90",
-        "matchPattern": "action order [0-9]*:  police 0x5a rate 3Mbit burst 250Kb mtu 2Kb action drop",
+        "matchPattern": "action order [0-9]*: police.*index 90 rate 3Mbit burst 250Kb mtu 2Kb action drop",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -783,7 +783,7 @@ 
         "cmdUnderTest": "$TC actions add action police pkts_rate 1000 pkts_burst 200 index 1",
         "expExitCode": "0",
         "verifyCmd": "$TC actions ls action police",
-        "matchPattern": "action order [0-9]*:  police 0x1 rate 0bit burst 0b mtu 4096Mb pkts_rate 1000 pkts_burst 200",
+        "matchPattern": "action order [0-9]*: police.*index 1 rate 0bit burst 0b mtu 4096Mb pkts_rate 1000 pkts_burst 200",
         "matchCount": "1",
         "teardown": [
             "$TC actions flush action police"
@@ -807,7 +807,7 @@ 
         "cmdUnderTest": "$TC actions add action police rate 1kbit burst 10k pkts_rate 1000 pkts_burst 200 index 1",
         "expExitCode": "255",
         "verifyCmd": "$TC actions ls action police",
-        "matchPattern": "action order [0-9]*:  police 0x1 ",
+        "matchPattern": "action order [0-9]*: police.*index 1 ",
         "matchCount": "0",
         "teardown": [
             "$TC actions flush action police"