git-lng-odp

git-lng-odp@lists.linaro.org

1240 discussions

[COMMIT] tag v1.23.0.0 created. v1.21.0.0-323-gd26fd80e7

by git＠git-us.linaro.org

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The tag, v1.23.0.0 has been created at d26fd80e75f6ff310afaf8c74458869a85f01934 (commit) - Log ----------------------------------------------------------------- commit d26fd80e75f6ff310afaf8c74458869a85f01934 Author: Matias Elo <matias.elo(a)nokia.com> Date: Tue Nov 12 10:15:25 2019 +0200 changelog: updates for odp v1.23.0.0 Signed-off-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen(a)nokia.com> ----------------------------------------------------------------------- hooks/post-receive --

6 years, 9 months

[COMMIT] branch master updated. v1.21.0.0-323-gd26fd80e7

by git＠git-us.linaro.org

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via d26fd80e75f6ff310afaf8c74458869a85f01934 (commit) from b8d4e0f27c0979cd4fb628318b10b57538d67fdf (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit d26fd80e75f6ff310afaf8c74458869a85f01934 Author: Matias Elo <matias.elo(a)nokia.com> Date: Tue Nov 12 10:15:25 2019 +0200 changelog: updates for odp v1.23.0.0 Signed-off-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen(a)nokia.com> diff --git a/CHANGELOG b/CHANGELOG index 855f12acb..f3b366d8f 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,3 +1,166 @@ +== OpenDataPlane (1.23.0.0) +=== Summary of Changes +The ODP API changes in this release are related to the classifier module. + +The implementation changes include bug fixes (classifier, socket pktio), +configurability improvements (inline timer), new packet segmentation +implementation, and performance improvements (pool). + +=== API +==== Classifier +The PMR term documentation was not explicit about endianness of value and mask +fields. Most terms assumed CPU endian, but terms with larger data sizes assumed +big endian (MAC, IPv6 address and custom). + +Term specification was harmonized so that all terms expect value/mask data to be +in big-endian format, have fixed size, and allow free memory alignment. Packet +length term is an exception to this as it does not represent a field in a +packet. + +Added new `ODP_PMR_CUSTOM_L3` term to match custom layer 3 protocol fields. PMR +offset refers to the start of layer 3 in the packet. Other PMR rules (e.g. L2 +classification rules) may precede custom L3 rules. + +=== Helper +Duplicate `ODPH_UNUSED` macro has been removed. `ODP_UNUSED` should be used +instead. + +=== Validation Tests +==== Common +Improved test result output when some tests are inactive (skipped due to missing +capability). Now inactive tests are listed by default in the test suite results. + +==== Classifier +Cleaned up validation test code. Classifier implementation missed reporting some +capabilities, tests didn't check those capabilities and they were tested. Source +IPv4 term test was missing. + +Added serial PMR test, which tests series of PMR and CoS: + +* From default CoS to dest IPv4 CoS +* From dest IPv4 CoS to dest UDP CoS + +Added parallel PMR test, which test serial and parallel PMR and CoS: + +* From default CoS to dest IPv4 CoS +* From dest IPv4 CoS to a parallel UDP CoS per destination port number + +==== Scheduler +Added new validation tests to verify that packet order is maintained when events +are enqueued and dequeued to/from a plain queue while maintaining atomic/ordered +scheduling context. + +==== Timer +Added private timer pool test, which creates a timer pool with the `priv` flag +set. The same thread calls schedule / queue_deq that created the pool. + +=== Example Applications +==== timer_perf +Added new test application to measure schedule call CPU cycle consumption with +various timer pool parameter combinations. This measurement is mostly +interesting with software-based timer implementations, where timers may be +polled from the schedule call. + +==== New odp_ping application +This application replies to IPv4 ping requests. It can be used to test +connectivity with standard ping utility. ARP table needs to be setup manually on +the sender side as the application does not reply to ARP requests. + +==== odp_classifier +Added source CoS name into the parameter list of `-p` option. This enables +linking classification rules together. When a source CoS is not defined, the +default CoS is used as the source CoS. + +Option `-p` parameter list format is now: + + <term>:<xxx>:<yyy>:<src_cos>:<dst_cos> + +Where `<src_cos>` is optional and number of "xxx", "yyy", etc. parameters is +term depend. + +Added `-v` option which prints received packet with CoS queue +name. This can be used for debugging classification rules. + +Added support for `ODP_PMR_VLAN_ID_0`, `ODP_PMR_ETHTYPE_0`, `ODP_PMR_CUSTOM_L3`, +`ODP_PMR_ETHTYPE_X`, `ODP_PMR_VLAN_ID_X`, `ODP_PMR_UDP_DPORT`, +`ODP_PMR_UDP_SPORT`, `ODP_PMR_TCP_DPORT`, and `ODP_PMR_TCP_SPORT` rules. + +==== timer_accuracy +Fix bug in timer start offset calculation. Test results were offset by current +local time value. + +Added delay and extra schedule calls to ensure that (software) timer +implementation has passed its initial state before setting up timers. + +Added `-f` option to control offset to the first timer. This can be used e.g. to +avoid best/worst case synchronization of timers to timer tick boundaries / +resolution. + +==== packet_dump +Added ICMP support. Print ICMP offset and ICMPv4 type/code. This helps to verify +that ping requests are received. + +==== bench_packet +Added missing tests for the following packet functions: + +* `odp_packet_data_seg_len()` +* `odp_packet_free_sp()` +* `odp_packet_from_event_multi()` +* `odp_packet_to_event_multi()` +* `odp_packet_subtype()` +* `odp_packet_parse()` +* `odp_packet_parse_multi()` + +=== Implementation +==== Classifier +Cleaned up implementation. Fixed endianness. + +==== Inline Timer +Cleaned up code. Fixed accuracy issue with high resolution (<100us) timer pools. +Added configuration option `inline_poll_interval_nsec` to select default poll +interval in nsec. + +Added configure file option for inline timer (`inline_thread_type`) to select if +control threads poll shared timer pools or not. Control threads still poll their +private pools. With this user can configure control and worker threads to use +separate timer pools. + +==== Packet I/O +Enable explicit pktio type definition. The pktio type is selected by adding +pktio_type prefix in front of device name separated by a colon +(<pktio_type>:<if_name>). + +The socket mmsg pktio implementation has been refactored and performance has +been improved. + +The IPC pktio implementation has been modified to use standard ODP internal +`ring_ptr_t` rings to implement IPC rings. This enables removing the duplicate +`_ring_t` implementation. + +==== Pool +Packet segmentation has been reimplemented using a linked list. The new +implementation is simpler and has the added benefit of greatly reducing +packet and buffer header sizes. + +=== Bug Fixes +==== Unnumbered Bugs / Issues +* Fixed socket mmap pktio throughput collapse under heavy traffic + +* Fixed missing config header in install directory + +* Include only ODP defines in autogenerated header files + +* Fixed GCC 9 `address-of-packed-member` warnings + +* Disable building static test applications without static ODP lib + +* Fixed a segfault in `odp_pktin_recv_mq_tmo()` implementation + +* Fixed classifier vlan match bugs and tests + +* Fixed classifier matching of parallel PMRs + + == OpenDataPlane (1.22.0.0) === Summary of Changes ODP v1.22.0.0 adds several smaller API changes mainly related to API ----------------------------------------------------------------------- Summary of changes: CHANGELOG | 163 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 163 insertions(+) hooks/post-receive --

6 years, 9 months

[COMMIT] branch master updated. v1.21.0.0-322-gb8d4e0f27

by git＠git-us.linaro.org

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via b8d4e0f27c0979cd4fb628318b10b57538d67fdf (commit) from 58e1522451ae4a4ce62acad5ea2c47d96dafe3cd (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit b8d4e0f27c0979cd4fb628318b10b57538d67fdf Author: Matias Elo <matias.elo(a)nokia.com> Date: Thu Nov 7 16:31:03 2019 +0200 validation: ipsec: memset ipsec_test_part structs to zero Explicitly initialize ipsec_test_part structs to zero to avoid "missing initializer for field" warnings with old compilers (Debian 8, GCC 4.9.2). Signed-off-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reported-by: Risto Teittinen <risto.teittinen(a)nokia-bell-labs.com> diff --git a/test/validation/api/ipsec/ipsec_test_in.c b/test/validation/api/ipsec/ipsec_test_in.c index 632e2ec8b..0e6497887 100644 --- a/test/validation/api/ipsec/ipsec_test_in.c +++ b/test/validation/api/ipsec/ipsec_test_in.c @@ -526,6 +526,9 @@ static void test_in_ipv4_ah_sha256_replay(void) { odp_ipsec_sa_param_t param; odp_ipsec_sa_t sa; + ipsec_test_part test_repl; + + memset(&test_repl, 0, sizeof(ipsec_test_part)); ipsec_sa_param_fill(&param, true, true, 123, NULL, @@ -550,15 +553,9 @@ static void test_in_ipv4_ah_sha256_replay(void) }, }; - ipsec_test_part test_repl = { - .pkt_in = &pkt_ipv4_icmp_0_ah_sha256_1, - .out_pkt = 1, - .out = { - { .status.warn.all = 0, - .status.error.antireplay = 1, - .pkt_out = NULL }, - }, - }; + test_repl.pkt_in = &pkt_ipv4_icmp_0_ah_sha256_1; + test_repl.out_pkt = 1; + test_repl.out[0].status.error.antireplay = 1; ipsec_test_part test_1235 = { .pkt_in = &pkt_ipv4_icmp_0_ah_sha256_1235, @@ -632,6 +629,9 @@ static void test_in_ipv4_esp_null_sha256_replay(void) { odp_ipsec_sa_param_t param; odp_ipsec_sa_t sa; + ipsec_test_part test_repl; + + memset(&test_repl, 0, sizeof(ipsec_test_part)); ipsec_sa_param_fill(&param, true, false, 123, NULL, @@ -656,15 +656,9 @@ static void test_in_ipv4_esp_null_sha256_replay(void) }, }; - ipsec_test_part test_repl = { - .pkt_in = &pkt_ipv4_icmp_0_esp_null_sha256_1, - .out_pkt = 1, - .out = { - { .status.warn.all = 0, - .status.error.antireplay = 1, - .pkt_out = NULL }, - }, - }; + test_repl.pkt_in = &pkt_ipv4_icmp_0_esp_null_sha256_1; + test_repl.out_pkt = 1; + test_repl.out[0].status.error.antireplay = 1; ipsec_test_part test_1235 = { .pkt_in = &pkt_ipv4_icmp_0_esp_null_sha256_1235, @@ -690,6 +684,9 @@ static void test_in_ipv4_ah_esp_pkt(void) { odp_ipsec_sa_param_t param; odp_ipsec_sa_t sa; + ipsec_test_part test; + + memset(&test, 0, sizeof(ipsec_test_part)); /* This test will not work properly inbound inline mode. * test_in_ipv4_ah_esp_pkt_lookup will be used instead. */ @@ -706,15 +703,9 @@ static void test_in_ipv4_ah_esp_pkt(void) CU_ASSERT_NOT_EQUAL_FATAL(ODP_IPSEC_SA_INVALID, sa); - ipsec_test_part test = { - .pkt_in = &pkt_ipv4_icmp_0_esp_null_sha256_1, - .out_pkt = 1, - .out = { - { .status.warn.all = 0, - .status.error.proto = 1, - .pkt_out = NULL }, - }, - }; + test.pkt_in = &pkt_ipv4_icmp_0_esp_null_sha256_1; + test.out_pkt = 1; + test.out[0].status.error.proto = 1; ipsec_check_in_one(&test, sa); @@ -725,6 +716,9 @@ static void test_in_ipv4_esp_ah_pkt(void) { odp_ipsec_sa_param_t param; odp_ipsec_sa_t sa; + ipsec_test_part test; + + memset(&test, 0, sizeof(ipsec_test_part)); /* This test will not work properly inbound inline mode. * test_in_ipv4_esp_ah_pkt_lookup will be used instead. */ @@ -741,15 +735,9 @@ static void test_in_ipv4_esp_ah_pkt(void) CU_ASSERT_NOT_EQUAL_FATAL(ODP_IPSEC_SA_INVALID, sa); - ipsec_test_part test = { - .pkt_in = &pkt_ipv4_icmp_0_ah_sha256_1, - .out_pkt = 1, - .out = { - { .status.warn.all = 0, - .status.error.proto = 1, - .pkt_out = NULL }, - }, - }; + test.pkt_in = &pkt_ipv4_icmp_0_ah_sha256_1; + test.out_pkt = 1; + test.out[0].status.error.proto = 1; ipsec_check_in_one(&test, sa); @@ -760,6 +748,9 @@ static void test_in_ipv4_ah_esp_pkt_lookup(void) { odp_ipsec_sa_param_t param; odp_ipsec_sa_t sa; + ipsec_test_part test; + + memset(&test, 0, sizeof(ipsec_test_part)); ipsec_sa_param_fill(&param, true, true, 123, NULL, @@ -771,16 +762,10 @@ static void test_in_ipv4_ah_esp_pkt_lookup(void) CU_ASSERT_NOT_EQUAL_FATAL(ODP_IPSEC_SA_INVALID, sa); - ipsec_test_part test = { - .pkt_in = &pkt_ipv4_icmp_0_esp_null_sha256_1, - .lookup = 1, - .out_pkt = 1, - .out = { - { .status.warn.all = 0, - .status.error.sa_lookup = 1, - .pkt_out = NULL }, - }, - }; + test.pkt_in = &pkt_ipv4_icmp_0_esp_null_sha256_1; + test.lookup = 1; + test.out_pkt = 1; + test.out[0].status.error.sa_lookup = 1; ipsec_check_in_one(&test, ODP_IPSEC_SA_INVALID); @@ -791,6 +776,9 @@ static void test_in_ipv4_esp_ah_pkt_lookup(void) { odp_ipsec_sa_param_t param; odp_ipsec_sa_t sa; + ipsec_test_part test; + + memset(&test, 0, sizeof(ipsec_test_part)); ipsec_sa_param_fill(&param, true, false, 123, NULL, @@ -802,16 +790,10 @@ static void test_in_ipv4_esp_ah_pkt_lookup(void) CU_ASSERT_NOT_EQUAL_FATAL(ODP_IPSEC_SA_INVALID, sa); - ipsec_test_part test = { - .pkt_in = &pkt_ipv4_icmp_0_ah_sha256_1, - .lookup = 1, - .out_pkt = 1, - .out = { - { .status.warn.all = 0, - .status.error.sa_lookup = 1, - .pkt_out = NULL }, - }, - }; + test.pkt_in = &pkt_ipv4_icmp_0_ah_sha256_1; + test.lookup = 1; + test.out_pkt = 1; + test.out[0].status.error.sa_lookup = 1; ipsec_check_in_one(&test, ODP_IPSEC_SA_INVALID); @@ -822,6 +804,9 @@ static void test_in_ipv4_ah_sha256_bad1(void) { odp_ipsec_sa_param_t param; odp_ipsec_sa_t sa; + ipsec_test_part test; + + memset(&test, 0, sizeof(ipsec_test_part)); ipsec_sa_param_fill(&param, true, true, 123, NULL, @@ -833,15 +818,9 @@ static void test_in_ipv4_ah_sha256_bad1(void) CU_ASSERT_NOT_EQUAL_FATAL(ODP_IPSEC_SA_INVALID, sa); - ipsec_test_part test = { - .pkt_in = &pkt_ipv4_icmp_0_ah_sha256_1_bad1, - .out_pkt = 1, - .out = { - { .status.warn.all = 0, - .status.error.auth = 1, - .pkt_out = NULL }, - }, - }; + test.pkt_in = &pkt_ipv4_icmp_0_ah_sha256_1_bad1; + test.out_pkt = 1; + test.out[0].status.error.auth = 1; ipsec_check_in_one(&test, sa); @@ -852,6 +831,9 @@ static void test_in_ipv4_ah_sha256_bad2(void) { odp_ipsec_sa_param_t param; odp_ipsec_sa_t sa; + ipsec_test_part test; + + memset(&test, 0, sizeof(ipsec_test_part)); ipsec_sa_param_fill(&param, true, true, 123, NULL, @@ -863,15 +845,9 @@ static void test_in_ipv4_ah_sha256_bad2(void) CU_ASSERT_NOT_EQUAL_FATAL(ODP_IPSEC_SA_INVALID, sa); - ipsec_test_part test = { - .pkt_in = &pkt_ipv4_icmp_0_ah_sha256_1_bad2, - .out_pkt = 1, - .out = { - { .status.warn.all = 0, - .status.error.auth = 1, - .pkt_out = NULL }, - }, - }; + test.pkt_in = &pkt_ipv4_icmp_0_ah_sha256_1_bad2; + test.out_pkt = 1; + test.out[0].status.error.auth = 1; ipsec_check_in_one(&test, sa); @@ -882,6 +858,9 @@ static void test_in_ipv4_esp_null_sha256_bad1(void) { odp_ipsec_sa_param_t param; odp_ipsec_sa_t sa; + ipsec_test_part test; + + memset(&test, 0, sizeof(ipsec_test_part)); ipsec_sa_param_fill(&param, true, false, 123, NULL, @@ -893,15 +872,9 @@ static void test_in_ipv4_esp_null_sha256_bad1(void) CU_ASSERT_NOT_EQUAL_FATAL(ODP_IPSEC_SA_INVALID, sa); - ipsec_test_part test = { - .pkt_in = &pkt_ipv4_icmp_0_esp_null_sha256_1_bad1, - .out_pkt = 1, - .out = { - { .status.warn.all = 0, - .status.error.auth = 1, - .pkt_out = NULL }, - }, - }; + test.pkt_in = &pkt_ipv4_icmp_0_esp_null_sha256_1_bad1; + test.out_pkt = 1; + test.out[0].status.error.auth = 1; ipsec_check_in_one(&test, sa); diff --git a/test/validation/api/ipsec/ipsec_test_out.c b/test/validation/api/ipsec/ipsec_test_out.c index 36de0cca1..0bfb9cff8 100644 --- a/test/validation/api/ipsec/ipsec_test_out.c +++ b/test/validation/api/ipsec/ipsec_test_out.c @@ -558,6 +558,11 @@ static void test_out_ipv4_ah_sha256_frag_check(void) { odp_ipsec_sa_param_t param; odp_ipsec_sa_t sa; + ipsec_test_part test; + ipsec_test_part test2; + + memset(&test, 0, sizeof(ipsec_test_part)); + memset(&test2, 0, sizeof(ipsec_test_part)); ipsec_sa_param_fill(&param, false, true, 123, NULL, @@ -571,28 +576,16 @@ static void test_out_ipv4_ah_sha256_frag_check(void) CU_ASSERT_NOT_EQUAL_FATAL(ODP_IPSEC_SA_INVALID, sa); - ipsec_test_part test = { - .pkt_in = &pkt_ipv4_icmp_0, - .out_pkt = 1, - .out = { - { .status.warn.all = 0, - .status.error.mtu = 1, - .pkt_out = NULL }, - }, - }; + test.pkt_in = &pkt_ipv4_icmp_0; + test.out_pkt = 1; + test.out[0].status.error.mtu = 1; - ipsec_test_part test2 = { - .pkt_in = &pkt_ipv4_icmp_0, - .num_opt = 1, - .opt = { .flag.frag_mode = 1, - .frag_mode = ODP_IPSEC_FRAG_DISABLED, }, - .out_pkt = 1, - .out = { - { .status.warn.all = 0, - .status.error.all = 0, - .pkt_out = &pkt_ipv4_icmp_0_ah_sha256_1 }, - }, - }; + test2.pkt_in = &pkt_ipv4_icmp_0; + test2.num_opt = 1; + test2.opt.flag.frag_mode = 1; + test2.opt.frag_mode = ODP_IPSEC_FRAG_DISABLED; + test2.out_pkt = 1; + test2.out[0].pkt_out = &pkt_ipv4_icmp_0_ah_sha256_1; ipsec_check_out_one(&test, sa); @@ -605,6 +598,9 @@ static void test_out_ipv4_ah_sha256_frag_check_2(void) { odp_ipsec_sa_param_t param; odp_ipsec_sa_t sa; + ipsec_test_part test; + + memset(&test, 0, sizeof(ipsec_test_part)); ipsec_sa_param_fill(&param, false, true, 123, NULL, @@ -618,15 +614,9 @@ static void test_out_ipv4_ah_sha256_frag_check_2(void) CU_ASSERT_NOT_EQUAL_FATAL(ODP_IPSEC_SA_INVALID, sa); - ipsec_test_part test = { - .pkt_in = &pkt_ipv4_icmp_0, - .out_pkt = 1, - .out = { - { .status.warn.all = 0, - .status.error.mtu = 1, - .pkt_out = NULL }, - }, - }; + test.pkt_in = &pkt_ipv4_icmp_0; + test.out_pkt = 1; + test.out[0].status.error.mtu = 1; ipsec_test_part test2 = { .pkt_in = &pkt_ipv4_icmp_0, @@ -651,6 +641,11 @@ static void test_out_ipv4_esp_null_sha256_frag_check(void) { odp_ipsec_sa_param_t param; odp_ipsec_sa_t sa; + ipsec_test_part test; + ipsec_test_part test2; + + memset(&test, 0, sizeof(ipsec_test_part)); + memset(&test2, 0, sizeof(ipsec_test_part)); ipsec_sa_param_fill(&param, false, false, 123, NULL, @@ -665,28 +660,16 @@ static void test_out_ipv4_esp_null_sha256_frag_check(void) CU_ASSERT_NOT_EQUAL_FATAL(ODP_IPSEC_SA_INVALID, sa); - ipsec_test_part test = { - .pkt_in = &pkt_ipv4_icmp_0, - .out_pkt = 1, - .out = { - { .status.warn.all = 0, - .status.error.mtu = 1, - .pkt_out = NULL }, - }, - }; + test.pkt_in = &pkt_ipv4_icmp_0; + test.out_pkt = 1; + test.out[0].status.error.mtu = 1; - ipsec_test_part test2 = { - .pkt_in = &pkt_ipv4_icmp_0, - .num_opt = 1, - .opt = { .flag.frag_mode = 1, - .frag_mode = ODP_IPSEC_FRAG_DISABLED, }, - .out_pkt = 1, - .out = { - { .status.warn.all = 0, - .status.error.all = 0, - .pkt_out = &pkt_ipv4_icmp_0_esp_null_sha256_1 }, - }, - }; + test2.pkt_in = &pkt_ipv4_icmp_0; + test2.num_opt = 1; + test2.opt.flag.frag_mode = 1; + test2.opt.frag_mode = ODP_IPSEC_FRAG_DISABLED; + test2.out_pkt = 1; + test2.out[0].pkt_out = &pkt_ipv4_icmp_0_esp_null_sha256_1; ipsec_check_out_one(&test, sa); @@ -699,6 +682,9 @@ static void test_out_ipv4_esp_null_sha256_frag_check_2(void) { odp_ipsec_sa_param_t param; odp_ipsec_sa_t sa; + ipsec_test_part test; + + memset(&test, 0, sizeof(ipsec_test_part)); ipsec_sa_param_fill(&param, false, false, 123, NULL, @@ -713,15 +699,9 @@ static void test_out_ipv4_esp_null_sha256_frag_check_2(void) CU_ASSERT_NOT_EQUAL_FATAL(ODP_IPSEC_SA_INVALID, sa); - ipsec_test_part test = { - .pkt_in = &pkt_ipv4_icmp_0, - .out_pkt = 1, - .out = { - { .status.warn.all = 0, - .status.error.mtu = 1, - .pkt_out = NULL }, - }, - }; + test.pkt_in = &pkt_ipv4_icmp_0; + test.out_pkt = 1; + test.out[0].status.error.mtu = 1; ipsec_test_part test2 = { .pkt_in = &pkt_ipv4_icmp_0, @@ -1011,6 +991,11 @@ static void test_out_dummy_esp_null_sha256_tun_ipv4(void) odp_ipsec_sa_t sa2; uint32_t src = IPV4ADDR(10, 0, 111, 2); uint32_t dst = IPV4ADDR(10, 0, 222, 2); + ipsec_test_part test; + ipsec_test_part test_empty; + + memset(&test, 0, sizeof(ipsec_test_part)); + memset(&test_empty, 0, sizeof(ipsec_test_part)); memset(&tunnel, 0, sizeof(odp_ipsec_tunnel_param_t)); tunnel.type = ODP_IPSEC_TUNNEL_IPV4; @@ -1043,35 +1028,21 @@ static void test_out_dummy_esp_null_sha256_tun_ipv4(void) CU_ASSERT_NOT_EQUAL_FATAL(ODP_IPSEC_SA_INVALID, sa2); - ipsec_test_part test = { - .pkt_in = &pkt_test_nodata, - .num_opt = 1, - .opt = { .flag.tfc_dummy = 1, - .tfc_pad_len = 16, }, - .out_pkt = 1, - .out = { - { .status.warn.all = 0, - .status.error.all = 0, - .l3_type = ODP_PROTO_L3_TYPE_IPV4, - .l4_type = ODP_PROTO_L4_TYPE_NO_NEXT, - .pkt_out = NULL }, - }, - }; - - ipsec_test_part test_empty = { - .pkt_in = &pkt_test_emtpy, - .num_opt = 1, - .opt = { .flag.tfc_dummy = 1, - .tfc_pad_len = 16, }, - .out_pkt = 1, - .out = { - { .status.warn.all = 0, - .status.error.all = 0, - .l3_type = ODP_PROTO_L3_TYPE_IPV4, - .l4_type = ODP_PROTO_L4_TYPE_NO_NEXT, - .pkt_out = NULL }, - }, - }; + test.pkt_in = &pkt_test_nodata; + test.num_opt = 1; + test.opt .flag.tfc_dummy = 1; + test.opt.tfc_pad_len = 16; + test.out_pkt = 1; + test.out[0].l3_type = ODP_PROTO_L3_TYPE_IPV4; + test.out[0].l4_type = ODP_PROTO_L4_TYPE_NO_NEXT; + + test_empty.pkt_in = &pkt_test_emtpy; + test_empty.num_opt = 1; + test_empty.opt.flag.tfc_dummy = 1; + test_empty.opt.tfc_pad_len = 16; + test_empty.out_pkt = 1; + test_empty.out[0].l3_type = ODP_PROTO_L3_TYPE_IPV4; + test_empty.out[0].l4_type = ODP_PROTO_L4_TYPE_NO_NEXT; ipsec_check_out_in_one(&test, sa, sa2); ipsec_check_out_in_one(&test_empty, sa, sa2); @@ -1086,6 +1057,8 @@ static void test_out_dummy_esp_null_sha256_tun_ipv6(void) odp_ipsec_sa_param_t param; odp_ipsec_sa_t sa; odp_ipsec_sa_t sa2; + ipsec_test_part test; + ipsec_test_part test_empty; uint8_t src[16] = { 0x20, 0x01, 0x0d, 0xb8, 0x00, 0x00, 0x00, 0x00, 0x02, 0x11, 0x43, 0xff, 0xfe, 0x4a, 0xd7, 0x0a, @@ -1095,6 +1068,9 @@ static void test_out_dummy_esp_null_sha256_tun_ipv6(void) 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x16, }; + memset(&test, 0, sizeof(ipsec_test_part)); + memset(&test_empty, 0, sizeof(ipsec_test_part)); + memset(&tunnel, 0, sizeof(odp_ipsec_tunnel_param_t)); tunnel.type = ODP_IPSEC_TUNNEL_IPV6; tunnel.ipv6.src_addr = src; @@ -1126,35 +1102,21 @@ static void test_out_dummy_esp_null_sha256_tun_ipv6(void) CU_ASSERT_NOT_EQUAL_FATAL(ODP_IPSEC_SA_INVALID, sa2); - ipsec_test_part test = { - .pkt_in = &pkt_test_nodata, - .num_opt = 1, - .opt = { .flag.tfc_dummy = 1, - .tfc_pad_len = 16, }, - .out_pkt = 1, - .out = { - { .status.warn.all = 0, - .status.error.all = 0, - .l3_type = ODP_PROTO_L3_TYPE_IPV4, - .l4_type = ODP_PROTO_L4_TYPE_NO_NEXT, - .pkt_out = NULL }, - }, - }; - - ipsec_test_part test_empty = { - .pkt_in = &pkt_test_emtpy, - .num_opt = 1, - .opt = { .flag.tfc_dummy = 1, - .tfc_pad_len = 16, }, - .out_pkt = 1, - .out = { - { .status.warn.all = 0, - .status.error.all = 0, - .l3_type = ODP_PROTO_L3_TYPE_IPV4, - .l4_type = ODP_PROTO_L4_TYPE_NO_NEXT, - .pkt_out = NULL }, - }, - }; + test.pkt_in = &pkt_test_nodata; + test.num_opt = 1; + test.opt .flag.tfc_dummy = 1; + test.opt.tfc_pad_len = 16; + test.out_pkt = 1; + test.out[0].l3_type = ODP_PROTO_L3_TYPE_IPV4; + test.out[0].l4_type = ODP_PROTO_L4_TYPE_NO_NEXT; + + test_empty.pkt_in = &pkt_test_emtpy; + test_empty.num_opt = 1; + test_empty.opt.flag.tfc_dummy = 1; + test_empty.opt.tfc_pad_len = 16; + test_empty.out_pkt = 1; + test_empty.out[0].l3_type = ODP_PROTO_L3_TYPE_IPV4; + test_empty.out[0].l4_type = ODP_PROTO_L4_TYPE_NO_NEXT; ipsec_check_out_in_one(&test, sa, sa2); ipsec_check_out_in_one(&test_empty, sa, sa2); ----------------------------------------------------------------------- Summary of changes: test/validation/api/ipsec/ipsec_test_in.c | 139 ++++++++------------ test/validation/api/ipsec/ipsec_test_out.c | 198 ++++++++++++----------------- 2 files changed, 136 insertions(+), 201 deletions(-) hooks/post-receive --

6 years, 9 months

[COMMIT] branch master updated. v1.21.0.0-321-g58e152245

by git＠git-us.linaro.org

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 58e1522451ae4a4ce62acad5ea2c47d96dafe3cd (commit) via fae79706192988fa1b89e68f11e3167c9495c771 (commit) from 3e3a204764ee055acb3f0ff0feff39a653362c38 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 58e1522451ae4a4ce62acad5ea2c47d96dafe3cd Author: Matias Elo <matias.elo(a)nokia.com> Date: Wed Nov 6 09:50:28 2019 +0200 test: bench_packet: add tests for packet parse functions Add new tests for odp_packet_parse() and odp_packet_parse_multi() functions using IPv4-TCP, IPv4-UDP, IPv6-TCP, and IPv6-UDP packets. The test packets are imported from the packet validation tests. Signed-off-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen(a)nokia.com> diff --git a/test/performance/odp_bench_packet.c b/test/performance/odp_bench_packet.c index 3bf61d86d..d139006b3 100644 --- a/test/performance/odp_bench_packet.c +++ b/test/performance/odp_bench_packet.c @@ -17,6 +17,8 @@ #include <inttypes.h> #include <signal.h> +#include <test_packet_parser.h> + #include <odp_api.h> #include <odp/helper/odph_api.h> @@ -356,6 +358,77 @@ static void alloc_packets_twice(void) TEST_REPEAT_COUNT); } +static void alloc_parse_packets(const void *pkt_data, uint32_t len) +{ + int i; + + allocate_test_packets(len, gbl_args->pkt_tbl, TEST_REPEAT_COUNT); + + for (i = 0; i < TEST_REPEAT_COUNT; i++) { + if (odp_packet_copy_from_mem(gbl_args->pkt_tbl[i], 0, len, + pkt_data)) + ODPH_ABORT("Copying test packet failed\n"); + } +} + +static void alloc_parse_packets_ipv4_tcp(void) +{ + alloc_parse_packets(test_packet_ipv4_tcp, sizeof(test_packet_ipv4_tcp)); +} + +static void alloc_parse_packets_ipv4_udp(void) +{ + alloc_parse_packets(test_packet_ipv4_udp, sizeof(test_packet_ipv4_udp)); +} + +static void alloc_parse_packets_ipv6_tcp(void) +{ + alloc_parse_packets(test_packet_ipv6_tcp, sizeof(test_packet_ipv6_tcp)); +} + +static void alloc_parse_packets_ipv6_udp(void) +{ + alloc_parse_packets(test_packet_ipv6_udp, sizeof(test_packet_ipv6_udp)); +} + +static void alloc_parse_packets_multi(const void *pkt_data, uint32_t len) +{ + int i; + + allocate_test_packets(len, gbl_args->pkt_tbl, + TEST_REPEAT_COUNT * gbl_args->appl.burst_size); + + for (i = 0; i < TEST_REPEAT_COUNT * gbl_args->appl.burst_size; i++) { + if (odp_packet_copy_from_mem(gbl_args->pkt_tbl[i], 0, len, + pkt_data)) + ODPH_ABORT("Copying test packet failed\n"); + } +} + +static void alloc_parse_packets_multi_ipv4_tcp(void) +{ + alloc_parse_packets_multi(test_packet_ipv4_tcp, + sizeof(test_packet_ipv4_tcp)); +} + +static void alloc_parse_packets_multi_ipv4_udp(void) +{ + alloc_parse_packets_multi(test_packet_ipv4_udp, + sizeof(test_packet_ipv4_udp)); +} + +static void alloc_parse_packets_multi_ipv6_tcp(void) +{ + alloc_parse_packets_multi(test_packet_ipv6_tcp, + sizeof(test_packet_ipv6_tcp)); +} + +static void alloc_parse_packets_multi_ipv6_udp(void) +{ + alloc_parse_packets_multi(test_packet_ipv6_udp, + sizeof(test_packet_ipv6_udp)); +} + static void create_packets(void) { int i; @@ -1352,6 +1425,53 @@ static int bench_packet_subtype(void) return i; } +static int bench_packet_parse(void) +{ + odp_packet_parse_param_t param; + odp_packet_t *pkt_tbl = gbl_args->pkt_tbl; + int ret = 0; + int i; + + memset(&param, 0, sizeof(odp_packet_parse_param_t)); + param.proto = ODP_PROTO_ETH; + param.last_layer = ODP_PROTO_LAYER_ALL; + param.chksums.chksum.ipv4 = 1; + param.chksums.chksum.tcp = 1; + param.chksums.chksum.udp = 1; + + for (i = 0; i < TEST_REPEAT_COUNT; i++) + ret += odp_packet_parse(pkt_tbl[i], 0, &param); + + return !ret; +} + +static int bench_packet_parse_multi(void) +{ + int burst_size = gbl_args->appl.burst_size; + int ret = 0; + int i; + odp_packet_parse_param_t param; + odp_packet_t *pkt_tbl = gbl_args->pkt_tbl; + uint32_t offsets[burst_size]; + + memset(&offsets, 0, sizeof(offsets)); + + memset(&param, 0, sizeof(odp_packet_parse_param_t)); + param.proto = ODP_PROTO_ETH; + param.last_layer = ODP_PROTO_LAYER_ALL; + param.chksums.chksum.ipv4 = 1; + param.chksums.chksum.tcp = 1; + param.chksums.chksum.udp = 1; + + for (i = 0; i < TEST_REPEAT_COUNT; i++) { + int idx = i * burst_size; + + ret += odp_packet_parse_multi(&pkt_tbl[idx], offsets, + burst_size, &param); + } + return (ret == TEST_REPEAT_COUNT * burst_size); +} + /** * Prinf usage information */ @@ -1586,6 +1706,30 @@ bench_info_t test_suite[] = { free_packets_twice, NULL), BENCH_INFO(bench_packet_subtype, create_packets, free_packets, NULL), + BENCH_INFO(bench_packet_parse, alloc_parse_packets_ipv4_tcp, + free_packets, "bench_packet_parse_ipv4_tcp"), + BENCH_INFO(bench_packet_parse, alloc_parse_packets_ipv4_udp, + free_packets, "bench_packet_parse_ipv4_udp"), + BENCH_INFO(bench_packet_parse, alloc_parse_packets_ipv6_tcp, + free_packets, "bench_packet_parse_ipv6_tcp"), + BENCH_INFO(bench_packet_parse, alloc_parse_packets_ipv6_udp, + free_packets, "bench_packet_parse_ipv6_udp"), + BENCH_INFO(bench_packet_parse_multi, + alloc_parse_packets_multi_ipv4_tcp, + free_packets_multi, + "bench_packet_parse_multi_ipv4_tcp"), + BENCH_INFO(bench_packet_parse_multi, + alloc_parse_packets_multi_ipv4_udp, + free_packets_multi, + "bench_packet_parse_multi_ipv4_udp"), + BENCH_INFO(bench_packet_parse_multi, + alloc_parse_packets_multi_ipv6_tcp, + free_packets_multi, + "bench_packet_parse_multi_ipv6_tcp"), + BENCH_INFO(bench_packet_parse_multi, + alloc_parse_packets_multi_ipv6_udp, + free_packets_multi, + "bench_packet_parse_multi_ipv6_udp"), }; /** commit fae79706192988fa1b89e68f11e3167c9495c771 Author: Matias Elo <matias.elo(a)nokia.com> Date: Wed Nov 6 09:50:03 2019 +0200 test: bench_packet: add tests for missing packet functions Added tests for the following packet functions: odp_packet_data_seg_len() odp_packet_free_sp() odp_packet_from_event_multi() odp_packet_to_event_multi() odp_packet_subtype() Signed-off-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen(a)nokia.com> diff --git a/test/performance/odp_bench_packet.c b/test/performance/odp_bench_packet.c index d050debb9..3bf61d86d 100644 --- a/test/performance/odp_bench_packet.c +++ b/test/performance/odp_bench_packet.c @@ -413,6 +413,18 @@ static void create_events(void) gbl_args->event_tbl[i] = odp_packet_to_event(pkt_tbl[i]); } +static void create_events_multi(void) +{ + int i; + odp_packet_t *pkt_tbl = gbl_args->pkt_tbl; + + allocate_test_packets(gbl_args->pkt.len, gbl_args->pkt_tbl, + TEST_REPEAT_COUNT * gbl_args->appl.burst_size); + + for (i = 0; i < TEST_REPEAT_COUNT * gbl_args->appl.burst_size; i++) + gbl_args->event_tbl[i] = odp_packet_to_event(pkt_tbl[i]); +} + static void free_packets(void) { odp_packet_free_multi(gbl_args->pkt_tbl, TEST_REPEAT_COUNT); @@ -491,6 +503,19 @@ static int bench_packet_free_multi(void) return i; } +static int bench_packet_free_sp(void) +{ + int i; + + for (i = 0; i < TEST_REPEAT_COUNT; i++) { + int pkt_idx = i * gbl_args->appl.burst_size; + + odp_packet_free_sp(&gbl_args->pkt_tbl[pkt_idx], + gbl_args->appl.burst_size); + } + return i; +} + static int bench_packet_alloc_free(void) { int i; @@ -545,6 +570,20 @@ static int bench_packet_from_event(void) return i; } +static int bench_packet_from_event_multi(void) +{ + int i; + + for (i = 0; i < TEST_REPEAT_COUNT; i++) { + int idx = i * gbl_args->appl.burst_size; + + odp_packet_from_event_multi(&gbl_args->pkt_tbl[idx], + &gbl_args->event_tbl[idx], + gbl_args->appl.burst_size); + } + return i; +} + static int bench_packet_to_event(void) { int i; @@ -556,6 +595,20 @@ static int bench_packet_to_event(void) return i; } +static int bench_packet_to_event_multi(void) +{ + int i; + + for (i = 0; i < TEST_REPEAT_COUNT; i++) { + int idx = i * gbl_args->appl.burst_size; + + odp_packet_to_event_multi(&gbl_args->pkt_tbl[idx], + &gbl_args->event_tbl[idx], + gbl_args->appl.burst_size); + } + return i; +} + static int bench_packet_head(void) { int i; @@ -587,6 +640,18 @@ static int bench_packet_data(void) return i; } +static int bench_packet_data_seg_len(void) +{ + odp_packet_t *pkt_tbl = gbl_args->pkt_tbl; + uint32_t *output_tbl = gbl_args->output_tbl; + int i; + + for (i = 0; i < TEST_REPEAT_COUNT; i++) + gbl_args->ptr_tbl[i] = odp_packet_data_seg_len(pkt_tbl[i], + &output_tbl[i]); + return i; +} + static int bench_packet_seg_len(void) { int i; @@ -1276,6 +1341,17 @@ static int bench_packet_has_ref(void) return i; } +static int bench_packet_subtype(void) +{ + int i; + odp_packet_t *pkt_tbl = gbl_args->pkt_tbl; + + for (i = 0; i < TEST_REPEAT_COUNT; i++) + gbl_args->output_tbl[i] = odp_packet_subtype(pkt_tbl[i]); + + return i; +} + /** * Prinf usage information */ @@ -1371,20 +1447,28 @@ bench_info_t test_suite[] = { BENCH_INFO(bench_packet_free, create_packets, NULL, NULL), BENCH_INFO(bench_packet_free_multi, alloc_packets_multi, NULL, NULL), + BENCH_INFO(bench_packet_free_sp, alloc_packets_multi, NULL, + NULL), BENCH_INFO(bench_packet_alloc_free, NULL, NULL, NULL), BENCH_INFO(bench_packet_alloc_free_multi, NULL, NULL, NULL), BENCH_INFO(bench_packet_reset, create_packets, free_packets, NULL), BENCH_INFO(bench_packet_from_event, create_events, free_packets, NULL), + BENCH_INFO(bench_packet_from_event_multi, create_events_multi, + free_packets_multi, NULL), BENCH_INFO(bench_packet_to_event, create_packets, free_packets, NULL), + BENCH_INFO(bench_packet_to_event_multi, alloc_packets_multi, + free_packets_multi, NULL), BENCH_INFO(bench_packet_head, create_packets, free_packets, NULL), BENCH_INFO(bench_packet_buf_len, create_packets, free_packets, NULL), BENCH_INFO(bench_packet_data, create_packets, free_packets, NULL), + BENCH_INFO(bench_packet_data_seg_len, create_packets, + free_packets, NULL), BENCH_INFO(bench_packet_seg_len, create_packets, free_packets, NULL), BENCH_INFO(bench_packet_len, create_packets, free_packets, @@ -1500,6 +1584,8 @@ bench_info_t test_suite[] = { free_packets_twice, NULL), BENCH_INFO(bench_packet_has_ref, alloc_ref_packets, free_packets_twice, NULL), + BENCH_INFO(bench_packet_subtype, create_packets, free_packets, + NULL), }; /** ----------------------------------------------------------------------- Summary of changes: test/performance/odp_bench_packet.c | 230 ++++++++++++++++++++++++++++++++++++ 1 file changed, 230 insertions(+) hooks/post-receive --

6 years, 9 months

[COMMIT] branch master updated. v1.21.0.0-319-g3e3a20476

by git＠git-us.linaro.org

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 3e3a204764ee055acb3f0ff0feff39a653362c38 (commit) via f21fc80503c72ebbd0e3286f708c7019567f8a8b (commit) via c9e8517c4945acf2c5215e0872779f00357649cf (commit) from f1e216da9011444789da716bbc258e530897da6d (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 3e3a204764ee055acb3f0ff0feff39a653362c38 Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Tue Nov 5 16:10:23 2019 +0200 linux-gen: event: inline odp_event_type_multi() Inline event_type_multi function. Event type functions are performance critical as those are typically used after every schedule call. Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> diff --git a/platform/linux-generic/include/odp/api/plat/event_inlines.h b/platform/linux-generic/include/odp/api/plat/event_inlines.h index fbecd2c14..43ee5e09a 100644 --- a/platform/linux-generic/include/odp/api/plat/event_inlines.h +++ b/platform/linux-generic/include/odp/api/plat/event_inlines.h @@ -18,11 +18,12 @@ extern const _odp_buffer_inline_offset_t _odp_buffer_inline_offset; /* Inline functions by default */ #define _ODP_INLINE static inline #define odp_event_type __odp_event_type + #define odp_event_type_multi __odp_event_type_multi #else #define _ODP_INLINE #endif -_ODP_INLINE odp_event_type_t odp_event_type(odp_event_t event) +static inline odp_event_type_t __odp_event_type_get(odp_event_t event) { int8_t type; odp_buffer_t buf = (odp_buffer_t)event; @@ -32,6 +33,27 @@ _ODP_INLINE odp_event_type_t odp_event_type(odp_event_t event) return (odp_event_type_t)type; } +_ODP_INLINE odp_event_type_t odp_event_type(odp_event_t event) +{ + return __odp_event_type_get(event); +} + +_ODP_INLINE int odp_event_type_multi(const odp_event_t event[], int num, + odp_event_type_t *type_out) +{ + int i; + odp_event_type_t type = __odp_event_type_get(event[0]); + + for (i = 1; i < num; i++) { + if (__odp_event_type_get(event[i]) != type) + break; + } + + *type_out = type; + + return i; +} + /** @endcond */ #endif diff --git a/platform/linux-generic/odp_event.c b/platform/linux-generic/odp_event.c index 6f6b3aa0b..677cd0dde 100644 --- a/platform/linux-generic/odp_event.c +++ b/platform/linux-generic/odp_event.c @@ -41,22 +41,6 @@ odp_event_type_t odp_event_types(odp_event_t event, return event_type; } -int odp_event_type_multi(const odp_event_t event[], int num, - odp_event_type_t *type_out) -{ - int i; - odp_event_type_t type = odp_event_type(event[0]); - - for (i = 1; i < num; i++) { - if (odp_event_type(event[i]) != type) - break; - } - - *type_out = type; - - return i; -} - uint32_t odp_event_flow_id(odp_event_t event) { return event_flow_id(event); commit f21fc80503c72ebbd0e3286f708c7019567f8a8b Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Tue Nov 5 15:51:46 2019 +0200 linux-gen: buffer: inline odp_buffer_addr() Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> diff --git a/platform/linux-generic/include/odp/api/plat/buffer_inline_types.h b/platform/linux-generic/include/odp/api/plat/buffer_inline_types.h index 29f49f8db..f59df6705 100644 --- a/platform/linux-generic/include/odp/api/plat/buffer_inline_types.h +++ b/platform/linux-generic/include/odp/api/plat/buffer_inline_types.h @@ -23,6 +23,7 @@ extern "C" { /* Buffer header field offsets for inline functions */ typedef struct _odp_buffer_inline_offset_t { uint16_t event_type; + uint16_t base_data; } _odp_buffer_inline_offset_t; diff --git a/platform/linux-generic/include/odp/api/plat/buffer_inlines.h b/platform/linux-generic/include/odp/api/plat/buffer_inlines.h index 89920510c..632ba0e69 100644 --- a/platform/linux-generic/include/odp/api/plat/buffer_inlines.h +++ b/platform/linux-generic/include/odp/api/plat/buffer_inlines.h @@ -21,6 +21,7 @@ extern const _odp_buffer_inline_offset_t _odp_buffer_inline_offset; #define _ODP_INLINE static inline #define odp_buffer_from_event __odp_buffer_from_event #define odp_buffer_to_event __odp_buffer_to_event + #define odp_buffer_addr __odp_buffer_addr #else #define _ODP_INLINE #endif @@ -35,6 +36,11 @@ _ODP_INLINE odp_event_t odp_buffer_to_event(odp_buffer_t buf) return (odp_event_t)buf; } +_ODP_INLINE void *odp_buffer_addr(odp_buffer_t buf) +{ + return _odp_buf_hdr_field(buf, void *, base_data); +} + /** @endcond */ #endif diff --git a/platform/linux-generic/odp_buffer.c b/platform/linux-generic/odp_buffer.c index 0001e598b..4c5e72640 100644 --- a/platform/linux-generic/odp_buffer.c +++ b/platform/linux-generic/odp_buffer.c @@ -20,18 +20,12 @@ /* Fill in buffer header field offsets for inline functions */ const _odp_buffer_inline_offset_t ODP_ALIGNED_CACHE _odp_buffer_inline_offset = { - .event_type = offsetof(odp_buffer_hdr_t, event_type) + .event_type = offsetof(odp_buffer_hdr_t, event_type), + .base_data = offsetof(odp_buffer_hdr_t, base_data) }; #include <odp/visibility_end.h> -void *odp_buffer_addr(odp_buffer_t buf) -{ - odp_buffer_hdr_t *hdr = buf_hdl_to_hdr(buf); - - return hdr->base_data; -} - uint32_t odp_buffer_size(odp_buffer_t buf) { odp_buffer_hdr_t *hdr = buf_hdl_to_hdr(buf); commit c9e8517c4945acf2c5215e0872779f00357649cf Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Tue Nov 5 15:28:16 2019 +0200 linux-gen: buffer: inline event handle conversions Inlined API functions which convert buffer handle to/from event handle. Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> diff --git a/platform/linux-generic/Makefile.am b/platform/linux-generic/Makefile.am index 0d15e53d9..58c3f4244 100644 --- a/platform/linux-generic/Makefile.am +++ b/platform/linux-generic/Makefile.am @@ -28,6 +28,7 @@ odpapiplatincludedir= $(includedir)/odp/api/plat odpapiplatinclude_HEADERS = \ include/odp/api/plat/atomic_inlines.h \ include/odp/api/plat/buffer_inline_types.h \ + include/odp/api/plat/buffer_inlines.h \ include/odp/api/plat/byteorder_inlines.h \ include/odp/api/plat/cpu_inlines.h \ include/odp/api/plat/event_inlines.h \ @@ -238,6 +239,7 @@ endif if ODP_ABI_COMPAT __LIB__libodp_linux_la_SOURCES += \ odp_atomic_api.c \ + odp_buffer_api.c \ odp_byteorder_api.c \ odp_cpu_api.c \ odp_event_api.c \ diff --git a/platform/linux-generic/include-abi/odp/api/abi/buffer.h b/platform/linux-generic/include-abi/odp/api/abi/buffer.h index 60292d502..8239e15da 100644 --- a/platform/linux-generic/include-abi/odp/api/abi/buffer.h +++ b/platform/linux-generic/include-abi/odp/api/abi/buffer.h @@ -28,6 +28,9 @@ typedef ODP_HANDLE_T(odp_buffer_t); #define ODP_BUFFER_INVALID _odp_cast_scalar(odp_buffer_t, 0) +/* Inlined functions for non-ABI compat mode */ +#include <odp/api/plat/buffer_inlines.h> + /** * @} */ diff --git a/platform/linux-generic/include/odp/api/plat/buffer_inlines.h b/platform/linux-generic/include/odp/api/plat/buffer_inlines.h new file mode 100644 index 000000000..89920510c --- /dev/null +++ b/platform/linux-generic/include/odp/api/plat/buffer_inlines.h @@ -0,0 +1,40 @@ +/* Copyright (c) 2019, Nokia + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#ifndef ODP_PLAT_BUFFER_INLINES_H_ +#define ODP_PLAT_BUFFER_INLINES_H_ + +#include <odp/api/abi/buffer.h> +#include <odp/api/abi/event.h> + +#include <odp/api/plat/buffer_inline_types.h> + +/** @cond _ODP_HIDE_FROM_DOXYGEN_ */ + +extern const _odp_buffer_inline_offset_t _odp_buffer_inline_offset; + +#ifndef _ODP_NO_INLINE + /* Inline functions by default */ + #define _ODP_INLINE static inline + #define odp_buffer_from_event __odp_buffer_from_event + #define odp_buffer_to_event __odp_buffer_to_event +#else + #define _ODP_INLINE +#endif + +_ODP_INLINE odp_buffer_t odp_buffer_from_event(odp_event_t ev) +{ + return (odp_buffer_t)ev; +} + +_ODP_INLINE odp_event_t odp_buffer_to_event(odp_buffer_t buf) +{ + return (odp_event_t)buf; +} + +/** @endcond */ + +#endif diff --git a/platform/linux-generic/odp_buffer.c b/platform/linux-generic/odp_buffer.c index f39f76a57..0001e598b 100644 --- a/platform/linux-generic/odp_buffer.c +++ b/platform/linux-generic/odp_buffer.c @@ -25,16 +25,6 @@ _odp_buffer_inline_offset = { #include <odp/visibility_end.h> -odp_buffer_t odp_buffer_from_event(odp_event_t ev) -{ - return (odp_buffer_t)ev; -} - -odp_event_t odp_buffer_to_event(odp_buffer_t buf) -{ - return (odp_event_t)buf; -} - void *odp_buffer_addr(odp_buffer_t buf) { odp_buffer_hdr_t *hdr = buf_hdl_to_hdr(buf); diff --git a/platform/linux-generic/odp_buffer_api.c b/platform/linux-generic/odp_buffer_api.c new file mode 100644 index 000000000..01f99b158 --- /dev/null +++ b/platform/linux-generic/odp_buffer_api.c @@ -0,0 +1,11 @@ +/* Copyright (c) 2019, Nokia + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#include <odp/api/buffer.h> + +/* Non-inlined functions for ABI compat mode */ +#define _ODP_NO_INLINE +#include <odp/api/plat/buffer_inlines.h> ----------------------------------------------------------------------- Summary of changes: platform/linux-generic/Makefile.am | 2 + .../linux-generic/include-abi/odp/api/abi/buffer.h | 3 ++ .../include/odp/api/plat/buffer_inline_types.h | 1 + .../include/odp/api/plat/buffer_inlines.h | 46 ++++++++++++++++++++++ .../include/odp/api/plat/event_inlines.h | 24 ++++++++++- platform/linux-generic/odp_buffer.c | 20 +--------- .../{odp_cpu_api.c => odp_buffer_api.c} | 6 +-- platform/linux-generic/odp_event.c | 16 -------- 8 files changed, 80 insertions(+), 38 deletions(-) create mode 100644 platform/linux-generic/include/odp/api/plat/buffer_inlines.h copy platform/linux-generic/{odp_cpu_api.c => odp_buffer_api.c} (59%) hooks/post-receive --

6 years, 10 months

[COMMIT] branch master updated. v1.21.0.0-316-gf1e216da9

by git＠git-us.linaro.org

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via f1e216da9011444789da716bbc258e530897da6d (commit) via f7caa49d2ba750d1fd29a12e288b441a6dac545f (commit) via f678476829f61a34d5d1e2b69b92c157786162e5 (commit) via c37655b8b74aa50825be8d58afa18d91b4a9a1fd (commit) via 642f27acf0c39ade24d6ea6551dde362206f4d4f (commit) via d2a9548017edaceec8cc682183690592c4c483b2 (commit) via 68e1cd81665c3b8deede1974d653593af85ec183 (commit) from b921639a9df6a0300ca1c50b5853a37241806366 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit f1e216da9011444789da716bbc258e530897da6d Author: Matias Elo <matias.elo(a)nokia.com> Date: Fri Nov 1 12:00:41 2019 +0200 validation: scheduler: add scheduled and plain queue tests Add new validation tests to verify that packet order is maintained when events are enqueued and dequeued to/from a plain queue while maintaining atomic/ordered scheduling context. Signed-off-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Stanislaw Kardach <skardach(a)marvell.com> Tested-by: Jussi Kerttula <jussi.kerttula(a)nokia.com> diff --git a/test/validation/api/scheduler/scheduler.c b/test/validation/api/scheduler/scheduler.c index da87deaef..ca92a3cf8 100644 --- a/test/validation/api/scheduler/scheduler.c +++ b/test/validation/api/scheduler/scheduler.c @@ -55,6 +55,8 @@ #define WAIT_TOLERANCE (150 * ODP_TIME_MSEC_IN_NS) #define WAIT_1MS_RETRIES 1000 +#define SCHED_AND_PLAIN_ROUNDS 10000 + /* Test global variables */ typedef struct { int num_workers; @@ -72,6 +74,10 @@ typedef struct { odp_queue_t handle; char name[ODP_QUEUE_NAME_LEN]; } chaos_q[CHAOS_NUM_QUEUES]; + struct { + odp_queue_t sched; + odp_queue_t plain; + } sched_and_plain_q; } test_globals_t; typedef struct { @@ -1780,6 +1786,229 @@ static void scheduler_test_ordered_lock(void) CU_ASSERT(drain_queues() == 0); } +static int sched_and_plain_thread(void *arg) +{ + odp_event_t ev1, ev2; + thread_args_t *args = (thread_args_t *)arg; + test_globals_t *globals = args->globals; + odp_queue_t sched_queue = globals->sched_and_plain_q.sched; + odp_queue_t plain_queue = globals->sched_and_plain_q.plain; + odp_schedule_sync_t sync = odp_queue_sched_type(sched_queue); + uint64_t i, wait; + + /* Wait for all threads to start */ + odp_barrier_wait(&globals->barrier); + + /* Run the test */ + wait = odp_schedule_wait_time(10 * ODP_TIME_MSEC_IN_NS); + for (i = 0; i < SCHED_AND_PLAIN_ROUNDS; i++) { + uint32_t rand_val; + + /* Dequeue events from scheduled and plain queues */ + ev1 = odp_schedule(NULL, wait); + if (ev1 == ODP_EVENT_INVALID) + continue; + + if (sync == ODP_SCHED_SYNC_ORDERED) + odp_schedule_order_lock(0); + + ev2 = odp_queue_deq(plain_queue); + CU_ASSERT_FATAL(ev2 != ODP_EVENT_INVALID); + + /* Add random delay to stress scheduler implementation */ + odp_random_data((uint8_t *)&rand_val, sizeof(rand_val), + ODP_RANDOM_BASIC); + odp_time_wait_ns(rand_val % ODP_TIME_USEC_IN_NS); + + /* Enqueue events back to the end of the queues */ + CU_ASSERT_FATAL(!odp_queue_enq(plain_queue, ev2)); + + if (sync == ODP_SCHED_SYNC_ORDERED) + odp_schedule_order_unlock(0); + + CU_ASSERT_FATAL(!odp_queue_enq(sched_queue, ev1)); + } + + /* Make sure scheduling context is released */ + odp_schedule_pause(); + while ((ev1 = odp_schedule(NULL, wait)) != ODP_EVENT_INVALID) + CU_ASSERT_FATAL(!odp_queue_enq(sched_queue, ev1)); + + /* Don't resume scheduling until all threads have finished */ + odp_barrier_wait(&globals->barrier); + odp_schedule_resume(); + + return 0; +} + +static void scheduler_test_sched_and_plain(odp_schedule_sync_t sync) +{ + thread_args_t *args; + test_globals_t *globals; + odp_queue_t sched_queue; + odp_queue_t plain_queue; + odp_pool_t pool; + odp_queue_param_t queue_param; + odp_pool_param_t pool_param; + odp_queue_capability_t queue_capa; + odp_schedule_capability_t sched_capa; + odp_shm_t shm; + odp_event_t ev; + uint32_t *buf_data; + uint32_t seq; + uint64_t wait = odp_schedule_wait_time(100 * ODP_TIME_MSEC_IN_NS); + uint32_t events_per_queue = BUFS_PER_QUEUE / 2; + uint32_t prev_seq; + int first; + + CU_ASSERT_FATAL(!odp_schedule_capability(&sched_capa)); + CU_ASSERT_FATAL(!odp_queue_capability(&queue_capa)) + + if (sync == ODP_SCHED_SYNC_ORDERED && + sched_capa.max_ordered_locks == 0) { + printf("\n NO ORDERED LOCKS. scheduler_test_ordered_and_plain skipped.\n"); + return; + } + + /* Set up the scheduling environment */ + shm = odp_shm_lookup(GLOBALS_SHM_NAME); + CU_ASSERT_FATAL(shm != ODP_SHM_INVALID); + globals = odp_shm_addr(shm); + CU_ASSERT_PTR_NOT_NULL_FATAL(globals); + + shm = odp_shm_lookup(SHM_THR_ARGS_NAME); + CU_ASSERT_FATAL(shm != ODP_SHM_INVALID); + args = odp_shm_addr(shm); + CU_ASSERT_PTR_NOT_NULL_FATAL(args); + args->globals = globals; + + /* Make sure all events fit to queues */ + if (sched_capa.max_queue_size && + sched_capa.max_queue_size < events_per_queue) + events_per_queue = sched_capa.max_queue_size; + if (queue_capa.plain.max_size && + queue_capa.plain.max_size < events_per_queue) + events_per_queue = queue_capa.plain.max_size; + + odp_queue_param_init(&queue_param); + queue_param.type = ODP_QUEUE_TYPE_SCHED; + queue_param.sched.sync = sync; + queue_param.size = events_per_queue; + if (sync == ODP_SCHED_SYNC_ORDERED) + queue_param.sched.lock_count = 1; + + sched_queue = odp_queue_create(NULL, &queue_param); + CU_ASSERT_FATAL(sched_queue != ODP_QUEUE_INVALID); + globals->sched_and_plain_q.sched = sched_queue; + + odp_queue_param_init(&queue_param); + queue_param.type = ODP_QUEUE_TYPE_PLAIN; + queue_param.size = events_per_queue; + + plain_queue = odp_queue_create(NULL, &queue_param); + CU_ASSERT_FATAL(sched_queue != ODP_QUEUE_INVALID); + globals->sched_and_plain_q.plain = plain_queue; + + odp_pool_param_init(&pool_param); + pool_param.buf.size = 100; + pool_param.buf.num = 2 * events_per_queue; + pool_param.type = ODP_POOL_BUFFER; + + pool = odp_pool_create("sched_to_plain_pool", &pool_param); + CU_ASSERT_FATAL(pool != ODP_POOL_INVALID); + + /* Create and enq test events with sequential sequence numbers */ + for (seq = 0; seq < events_per_queue; seq++) { + odp_buffer_t buf1, buf2; + + buf1 = odp_buffer_alloc(pool); + if (buf1 == ODP_BUFFER_INVALID) + break; + buf2 = odp_buffer_alloc(pool); + if (buf2 == ODP_BUFFER_INVALID) { + odp_buffer_free(buf1); + break; + } + buf_data = odp_buffer_addr(buf1); + *buf_data = seq; + buf_data = odp_buffer_addr(buf2); + *buf_data = seq; + + /* Events flow id is 0 by default */ + CU_ASSERT_FATAL(!odp_queue_enq(sched_queue, + odp_buffer_to_event(buf1))); + CU_ASSERT_FATAL(!odp_queue_enq(plain_queue, + odp_buffer_to_event(buf2))); + } + CU_ASSERT_FATAL(seq > 2); + + /* Test runs also on the main thread */ + args->cu_thr.numthrds = globals->num_workers - 1; + if (args->cu_thr.numthrds > 0) + odp_cunit_thread_create(sched_and_plain_thread, &args->cu_thr); + + sched_and_plain_thread(args); + + if (args->cu_thr.numthrds > 0) + odp_cunit_thread_exit(&args->cu_thr); + + /* Check plain queue sequence numbers and free events */ + first = 1; + while (1) { + ev = odp_queue_deq(plain_queue); + if (ev == ODP_EVENT_INVALID) + break; + + buf_data = odp_buffer_addr(odp_buffer_from_event(ev)); + seq = *buf_data; + + if (first) { + first = 0; + prev_seq = seq; + continue; + } + + CU_ASSERT(seq == prev_seq + 1 || seq == 0) + prev_seq = seq; + odp_event_free(ev); + } + + /* Check scheduled queue sequence numbers and free events */ + first = 1; + while (1) { + ev = odp_schedule(NULL, wait); + if (ev == ODP_EVENT_INVALID) + break; + + buf_data = odp_buffer_addr(odp_buffer_from_event(ev)); + seq = *buf_data; + + if (first) { + first = 0; + prev_seq = seq; + continue; + } + + CU_ASSERT(seq == prev_seq + 1 || seq == 0) + prev_seq = seq; + odp_event_free(ev); + } + + CU_ASSERT(!odp_queue_destroy(sched_queue)); + CU_ASSERT(!odp_queue_destroy(plain_queue)); + CU_ASSERT(!odp_pool_destroy(pool)); +} + +static void scheduler_test_atomic_and_plain(void) +{ + scheduler_test_sched_and_plain(ODP_SCHED_SYNC_ATOMIC); +} + +static void scheduler_test_ordered_and_plain(void) +{ + scheduler_test_sched_and_plain(ODP_SCHED_SYNC_ORDERED); +} + static int create_queues(test_globals_t *globals) { int i, j, prios, rc; @@ -2278,6 +2507,8 @@ odp_testinfo_t scheduler_suite[] = { ODP_TEST_INFO(scheduler_test_parallel), ODP_TEST_INFO(scheduler_test_atomic), ODP_TEST_INFO(scheduler_test_ordered), + ODP_TEST_INFO(scheduler_test_atomic_and_plain), + ODP_TEST_INFO(scheduler_test_ordered_and_plain), ODP_TEST_INFO(scheduler_test_chaos), ODP_TEST_INFO(scheduler_test_1q_1t_n), ODP_TEST_INFO(scheduler_test_1q_1t_a), commit f7caa49d2ba750d1fd29a12e288b441a6dac545f Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Thu Oct 31 16:42:19 2019 +0200 test: timer_perf: add private timer pool support Added -s option to select between shared and private timer pools. Currently, private timer pools can be tested only with single CPU (the master thread runs the test). Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> diff --git a/test/performance/odp_timer_perf.c b/test/performance/odp_timer_perf.c index d0f936a8e..c49baf0bd 100644 --- a/test/performance/odp_timer_perf.c +++ b/test/performance/odp_timer_perf.c @@ -25,6 +25,7 @@ typedef struct test_options_t { uint32_t num_timer; uint64_t res_ns; uint64_t period_ns; + int shared; } test_options_t; @@ -87,6 +88,10 @@ static void print_usage(void) " -t, --num_timer Number of timers per timer pool. Default: 10\n" " -r, --res_ns Resolution in nsec. Default: 10000000\n" " -p, --period_ns Timeout period in nsec. Default: 100000000\n" + " -s, --shared Shared vs private timer pool. Currently, private pools can be\n" + " tested only with single CPU. Default: 1\n" + " 0: Private timer pools\n" + " 1: Shared timer pools\n" " -h, --help This help\n" "\n"); } @@ -103,17 +108,19 @@ static int parse_options(int argc, char *argv[], test_options_t *test_options) {"num_timer", required_argument, NULL, 't'}, {"res_ns", required_argument, NULL, 'r'}, {"period_ns", required_argument, NULL, 'p'}, + {"shared", required_argument, NULL, 's'}, {"help", no_argument, NULL, 'h'}, {NULL, 0, NULL, 0} }; - static const char *shortopts = "+c:n:t:r:p:h"; + static const char *shortopts = "+c:n:t:r:p:s:h"; test_options->num_cpu = 1; test_options->num_tp = 1; test_options->num_timer = 10; test_options->res_ns = 10 * ODP_TIME_MSEC_IN_NS; test_options->period_ns = 100 * ODP_TIME_MSEC_IN_NS; + test_options->shared = 1; while (1) { opt = getopt_long(argc, argv, shortopts, longopts, &long_index); @@ -137,6 +144,9 @@ static int parse_options(int argc, char *argv[], test_options_t *test_options) case 'p': test_options->period_ns = atoll(optarg); break; + case 's': + test_options->shared = atoi(optarg); + break; case 'h': /* fall through */ default: @@ -159,6 +169,7 @@ static int set_num_cpu(test_global_t *global) int ret; test_options_t *test_options = &global->test_options; int num_cpu = test_options->num_cpu; + int shared = test_options->shared; /* One thread used for the main thread */ if (num_cpu > ODP_THREAD_COUNT_MAX - 1) { @@ -174,14 +185,21 @@ static int set_num_cpu(test_global_t *global) return -1; } + if (shared == 0 && num_cpu != 1) { + printf("Error: Private pool test supports only single CPU\n."); + return -1; + } + /* Zero: all available workers */ if (num_cpu == 0) { num_cpu = ret; test_options->num_cpu = num_cpu; } - /* Main thread + all workers */ - odp_barrier_init(&global->barrier, num_cpu + 1); + if (shared) /* Main thread + all workers */ + odp_barrier_init(&global->barrier, num_cpu + 1); + else /* Only the main thread */ + odp_barrier_init(&global->barrier, 1); return 0; } @@ -204,11 +222,17 @@ static int create_timer_pools(test_global_t *global) uint32_t num_timer = test_options->num_timer; uint64_t res_ns = test_options->res_ns; uint64_t period_ns = test_options->period_ns; + int priv; max_tmo_ns = START_NS + (num_timer * period_ns); + priv = 0; + if (test_options->shared == 0) + priv = 1; + printf("\nTimer performance test\n"); printf(" num cpu %u\n", num_cpu); + printf(" private pool %i\n", priv); printf(" num timer pool %u\n", num_tp); printf(" num timer %u\n", num_timer); printf(" resolution %" PRIu64 " nsec\n", res_ns); @@ -266,7 +290,7 @@ static int create_timer_pools(test_global_t *global) timer_pool_param.min_tmo = START_NS; timer_pool_param.max_tmo = max_tmo_ns; timer_pool_param.num_timers = num_timer; - timer_pool_param.priv = 0; + timer_pool_param.priv = priv; timer_pool_param.clk_src = ODP_CLOCK_CPU; odp_pool_param_init(&pool_param); @@ -540,8 +564,6 @@ static int start_workers(test_global_t *global, odp_instance_t instance) thr_param[i].start = test_worker; thr_param[i].arg = &global->thread_arg[i]; thr_param[i].thr_type = ODP_THREAD_WORKER; - - global->thread_arg[i].global = global; } ret = odph_thread_create(global->thread_tbl, &thr_common, thr_param, @@ -644,16 +666,24 @@ int main(int argc, char **argv) odp_instance_t instance; odp_init_t init; test_global_t *global; + test_options_t *test_options; + int i, shared; global = &test_global; memset(global, 0, sizeof(test_global_t)); odp_atomic_init_u32(&global->exit_test, 0); + for (i = 0; i < ODP_THREAD_COUNT_MAX; i++) + global->thread_arg[i].global = global; + signal(SIGINT, sig_handler); if (parse_options(argc, argv, &global->test_options)) return -1; + test_options = &global->test_options; + shared = test_options->shared; + /* List features not to be used */ odp_init_param_init(&init); init.not_used.feat.cls = 1; @@ -681,22 +711,33 @@ int main(int argc, char **argv) if (create_timer_pools(global)) return -1; - /* Start worker threads */ - start_workers(global, instance); + if (shared) { + /* Start worker threads */ + start_workers(global, instance); - /* Wait until workers have started. - * Scheduler calls from workers may be needed to run timer pools in - * a software implementation. Wait 1 msec to ensure that timer pools - * are running before setting timers. */ - odp_barrier_wait(&global->barrier); - odp_time_wait_ns(ODP_TIME_MSEC_IN_NS); + /* Wait until workers have started. + * Scheduler calls from workers may be needed to run timer + * pools in a software implementation. Wait 1 msec to ensure + * that timer pools are running before setting timers. */ + odp_barrier_wait(&global->barrier); + odp_time_wait_ns(ODP_TIME_MSEC_IN_NS); + } /* Set timers. Force workers to exit on failure. */ if (set_timers(global)) odp_atomic_add_u32(&global->exit_test, MAX_TIMER_POOLS); - /* Wait workers to exit */ - odph_thread_join(global->thread_tbl, global->test_options.num_cpu); + if (!shared) { + /* Test private pools on the master thread */ + if (test_worker(&global->thread_arg[0])) { + printf("Error: test loop failed\n"); + return -1; + } + } else { + /* Wait workers to exit */ + odph_thread_join(global->thread_tbl, + global->test_options.num_cpu); + } print_stat(global); commit f678476829f61a34d5d1e2b69b92c157786162e5 Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Thu Oct 31 10:27:02 2019 +0200 validation: timer: add private timer pool test Create timer pool with the 'priv' flag set. The same thread calls schedule / queue_deq that created the pool. Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> diff --git a/test/validation/api/timer/timer.c b/test/validation/api/timer/timer.c index 376a1218b..c9a70205e 100644 --- a/test/validation/api/timer/timer.c +++ b/test/validation/api/timer/timer.c @@ -536,7 +536,7 @@ static void timer_test_pkt_event_sched(void) timer_test_event_type(ODP_QUEUE_TYPE_SCHED, ODP_EVENT_PACKET); } -static void timer_test_queue_type(odp_queue_type_t queue_type) +static void timer_test_queue_type(odp_queue_type_t queue_type, int priv) { odp_pool_t pool; const int num = 10; @@ -567,7 +567,7 @@ static void timer_test_queue_type(odp_queue_type_t queue_type) tparam.min_tmo = global_mem->param.min_tmo; tparam.max_tmo = global_mem->param.max_tmo; tparam.num_timers = num + 1; - tparam.priv = 0; + tparam.priv = priv; tparam.clk_src = ODP_CLOCK_CPU; ODPH_DBG("\nTimer pool parameters:\n"); @@ -665,8 +665,7 @@ static void timer_test_queue_type(odp_queue_type_t queue_type) CU_ASSERT(diff_test > (test_period - period_ns)); CU_ASSERT(diff_test < (test_period + period_ns)); - /* Scalable scheduler needs this pause sequence. Otherwise, it gets - * stuck on terminate. */ + /* Reset scheduler context for the next test case */ if (queue_type == ODP_QUEUE_TYPE_SCHED) { odp_schedule_pause(); while (1) { @@ -677,6 +676,7 @@ static void timer_test_queue_type(odp_queue_type_t queue_type) CU_FAIL("Drop extra event\n"); odp_event_free(ev); } + odp_schedule_resume(); } odp_timer_pool_destroy(tp); @@ -687,12 +687,22 @@ static void timer_test_queue_type(odp_queue_type_t queue_type) static void timer_test_plain_queue(void) { - timer_test_queue_type(ODP_QUEUE_TYPE_PLAIN); + timer_test_queue_type(ODP_QUEUE_TYPE_PLAIN, 0); } static void timer_test_sched_queue(void) { - timer_test_queue_type(ODP_QUEUE_TYPE_SCHED); + timer_test_queue_type(ODP_QUEUE_TYPE_SCHED, 0); +} + +static void timer_test_plain_queue_priv(void) +{ + timer_test_queue_type(ODP_QUEUE_TYPE_PLAIN, 1); +} + +static void timer_test_sched_queue_priv(void) +{ + timer_test_queue_type(ODP_QUEUE_TYPE_SCHED, 1); } static void timer_test_cancel(void) @@ -1451,6 +1461,8 @@ odp_testinfo_t timer_suite[] = { ODP_TEST_INFO(timer_test_max_tmo_max_tmo_sched), ODP_TEST_INFO(timer_test_plain_queue), ODP_TEST_INFO(timer_test_sched_queue), + ODP_TEST_INFO(timer_test_plain_queue_priv), + ODP_TEST_INFO(timer_test_sched_queue_priv), ODP_TEST_INFO(timer_test_all), ODP_TEST_INFO_NULL, }; commit c37655b8b74aa50825be8d58afa18d91b4a9a1fd Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Wed Oct 30 16:08:34 2019 +0200 linux-gen: timer: use thread type config option Use configure file option to select if control threads poll shared timer pools. Timer processing of control and worker threads may be separated completely, if control threads create private timer pools and do not process shared pools. This way, worker threads will not poll control thread pools and vice versa, but both can use timers. Implemented private timer pools with minimal effort. Implementation of private and shared pools could be separated in the future (for better performance), as private pools do not need data synchronization between other threads. Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> diff --git a/platform/linux-generic/odp_timer.c b/platform/linux-generic/odp_timer.c index e82d29059..3a82ee84b 100644 --- a/platform/linux-generic/odp_timer.c +++ b/platform/linux-generic/odp_timer.c @@ -127,6 +127,7 @@ typedef struct timer_pool_s { odp_shm_t shm; timer_t timerid; int notify_overrun; + int owner; pthread_t thr_pthread; /* pthread_t of timer thread */ pid_t thr_pid; /* gettid() for timer thread */ int thr_warm_up; /* number of warm up rounds */ @@ -167,6 +168,7 @@ typedef struct timer_global_t { typedef struct timer_local_t { odp_time_t last_run; int run_cnt; + uint8_t poll_shared; } timer_local_t; @@ -353,6 +355,11 @@ static odp_timer_pool_t timer_pool_new(const char *name, odp_atomic_init_u32(&tp->high_wm, 0); tp->first_free = 0; tp->notify_overrun = 1; + tp->owner = -1; + + if (param->priv) + tp->owner = odp_thread_id(); + tp->tick_buf = (void *)((char *)odp_shm_addr(shm) + sz0); tp->timers = (void *)((char *)odp_shm_addr(shm) + sz0 + sz1); @@ -922,6 +929,17 @@ static inline void timer_pool_scan_inline(int num, odp_time_t now) if (tp == NULL) continue; + if (odp_likely(tp->owner < 0)) { + /* Skip shared pool, if this thread is not configured + * to process those */ + if (odp_unlikely(timer_local.poll_shared == 0)) + continue; + } else { + /* Skip private pool, if this thread is not the owner */ + if (tp->owner != odp_thread_id()) + continue; + } + nsec = time_nsec(tp, now); new_tick = nsec / tp->nsec_per_scan; old_tick = odp_atomic_load_u64(&tp->cur_tick); @@ -1557,8 +1575,27 @@ int _odp_timer_term_global(void) int _odp_timer_init_local(void) { + int conf_thr_type; + odp_thread_type_t thr_type; + timer_local.last_run = odp_time_global_from_ns(0); timer_local.run_cnt = 1; + timer_local.poll_shared = 0; + + /* Timer feature disabled */ + if (timer_global == NULL) + return 0; + + /* Check if this thread polls shared (non-private) timer pools */ + conf_thr_type = timer_global->thread_type; + thr_type = odp_thread_type(); + + if (conf_thr_type == 0) + timer_local.poll_shared = 1; + else if (conf_thr_type == 1 && thr_type == ODP_THREAD_WORKER) + timer_local.poll_shared = 1; + else if (conf_thr_type == 2 && thr_type == ODP_THREAD_CONTROL) + timer_local.poll_shared = 1; return 0; } commit 642f27acf0c39ade24d6ea6551dde362206f4d4f Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Wed Oct 30 13:57:23 2019 +0200 linux-gen: timer: move thread local variables into struct Moved type definition into beginning of the file and gathered thread local variables into timer_local_t struct. Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> diff --git a/platform/linux-generic/odp_timer.c b/platform/linux-generic/odp_timer.c index 4af31bcc1..e82d29059 100644 --- a/platform/linux-generic/odp_timer.c +++ b/platform/linux-generic/odp_timer.c @@ -66,35 +66,12 @@ /* Max inline timer resolution */ #define MAX_INLINE_RES_NS 500 -/****************************************************************************** - * Mutual exclusion in the absence of CAS16 - *****************************************************************************/ - +/* Mutual exclusion in the absence of CAS16 */ #ifndef ODP_ATOMIC_U128 #define NUM_LOCKS 1024 #define IDX2LOCK(idx) (&timer_global->locks[(idx) % NUM_LOCKS]) #endif -/****************************************************************************** - * Translation between timeout buffer and timeout header - *****************************************************************************/ - -static odp_timeout_hdr_t *timeout_hdr_from_buf(odp_buffer_t buf) -{ - return (odp_timeout_hdr_t *)(void *)buf_hdl_to_hdr(buf); -} - -static odp_timeout_hdr_t *timeout_hdr(odp_timeout_t tmo) -{ - odp_buffer_t buf = odp_buffer_from_event(odp_timeout_to_event(tmo)); - - return timeout_hdr_from_buf(buf); -} - -/****************************************************************************** - * odp_timer abstract datatype - *****************************************************************************/ - typedef struct #ifdef ODP_ATOMIC_U128 ODP_ALIGNED(16) /* 16-byte atomic operations need properly aligned addresses */ @@ -128,55 +105,11 @@ ODP_STATIC_ASSERT(sizeof(tick_buf_t) == 16, "sizeof(tick_buf_t) == 16"); typedef struct { void *user_ptr; odp_queue_t queue;/* Used for free list when timer is free */ -} _odp_timer_t; - -static void timer_init(_odp_timer_t *tim, - tick_buf_t *tb, - odp_queue_t _q, - void *_up) -{ - tim->queue = _q; - tim->user_ptr = _up; - tb->tmo_u64 = 0; - tb->tmo_buf = ODP_BUFFER_INVALID; - /* Release the timer by setting timer state to inactive */ -#if __GCC_ATOMIC_LLONG_LOCK_FREE < 2 - tb->exp_tck.v = TMO_INACTIVE; -#else - _odp_atomic_u64_store_mm(&tb->exp_tck, TMO_INACTIVE, _ODP_MEMMODEL_RLS); -#endif -} - -/* Teardown when timer is freed */ -static void timer_fini(_odp_timer_t *tim, tick_buf_t *tb) -{ - ODP_ASSERT(tb->exp_tck.v == TMO_UNUSED); - ODP_ASSERT(tb->tmo_buf == ODP_BUFFER_INVALID); - tim->queue = ODP_QUEUE_INVALID; - tim->user_ptr = NULL; -} - -static inline uint32_t get_next_free(_odp_timer_t *tim) -{ - /* Reusing 'queue' for next free index */ - return _odp_typeval(tim->queue); -} - -static inline void set_next_free(_odp_timer_t *tim, uint32_t nf) -{ - ODP_ASSERT(tim->queue == ODP_QUEUE_INVALID); - /* Reusing 'queue' for next free index */ - tim->queue = _odp_cast_scalar(odp_queue_t, nf); -} - -/****************************************************************************** - * timer_pool_t abstract datatype - * Inludes alloc and free timer - *****************************************************************************/ +} _odp_timer_t; typedef struct timer_pool_s { -/* Put frequently accessed fields in the first cache line */ + /* Put frequently accessed fields in the first cache line */ uint64_t nsec_per_scan; odp_time_t start_time; odp_atomic_u64_t cur_tick;/* Current tick value */ @@ -199,6 +132,7 @@ typedef struct timer_pool_s { int thr_warm_up; /* number of warm up rounds */ odp_atomic_u32_t thr_ready; /* thread ready from warm up */ int thr_exit; /* request to exit for timer thread */ + } timer_pool_t; /* Timer pool index must fit into 8 bits with one index value reserved to @@ -230,8 +164,62 @@ typedef struct timer_global_t { } timer_global_t; +typedef struct timer_local_t { + odp_time_t last_run; + int run_cnt; + +} timer_local_t; + +/* Points to timer global data */ static timer_global_t *timer_global; +/* Timer thread local data */ +static __thread timer_local_t timer_local; + +/* Forward declarations */ +static void itimer_init(timer_pool_t *tp); +static void itimer_fini(timer_pool_t *tp); + +static void timer_init(_odp_timer_t *tim, + tick_buf_t *tb, + odp_queue_t _q, + void *_up) +{ + tim->queue = _q; + tim->user_ptr = _up; + tb->tmo_u64 = 0; + tb->tmo_buf = ODP_BUFFER_INVALID; + + /* Release the timer by setting timer state to inactive */ +#if __GCC_ATOMIC_LLONG_LOCK_FREE < 2 + tb->exp_tck.v = TMO_INACTIVE; +#else + _odp_atomic_u64_store_mm(&tb->exp_tck, TMO_INACTIVE, _ODP_MEMMODEL_RLS); +#endif +} + +/* Teardown when timer is freed */ +static void timer_fini(_odp_timer_t *tim, tick_buf_t *tb) +{ + ODP_ASSERT(tb->exp_tck.v == TMO_UNUSED); + ODP_ASSERT(tb->tmo_buf == ODP_BUFFER_INVALID); + tim->queue = ODP_QUEUE_INVALID; + tim->user_ptr = NULL; +} + +static inline uint32_t get_next_free(_odp_timer_t *tim) +{ + /* Reusing 'queue' for next free index */ + return _odp_typeval(tim->queue); +} + +static inline void set_next_free(_odp_timer_t *tim, uint32_t nf) +{ + ODP_ASSERT(tim->queue == ODP_QUEUE_INVALID); + /* Reusing 'queue' for next free index */ + tim->queue = _odp_cast_scalar(odp_queue_t, nf); +} + static inline timer_pool_t *timer_pool_from_hdl(odp_timer_pool_t hdl) { return (timer_pool_t *)(uintptr_t)hdl; @@ -274,9 +262,17 @@ static inline odp_timer_t tp_idx_to_handle(timer_pool_t *tp, (idx + 1)); } -/* Forward declarations */ -static void itimer_init(timer_pool_t *tp); -static void itimer_fini(timer_pool_t *tp); +static odp_timeout_hdr_t *timeout_hdr_from_buf(odp_buffer_t buf) +{ + return (odp_timeout_hdr_t *)(void *)buf_hdl_to_hdr(buf); +} + +static odp_timeout_hdr_t *timeout_hdr(odp_timeout_t tmo) +{ + odp_buffer_t buf = odp_buffer_from_event(odp_timeout_to_event(tmo)); + + return timeout_hdr_from_buf(buf); +} static odp_timer_pool_t timer_pool_new(const char *name, const odp_timer_pool_param_t *param) @@ -952,8 +948,6 @@ static inline void timer_pool_scan_inline(int num, odp_time_t now) void _timer_run_inline(int dec) { - static __thread odp_time_t last_timer_run; - static __thread int timer_run_cnt = 1; odp_time_t now; int num = timer_global->highest_tp_idx + 1; int poll_interval = timer_global->poll_interval; @@ -964,21 +958,21 @@ void _timer_run_inline(int dec) /* Rate limit how often this thread checks the timer pools. */ if (poll_interval > 1) { - timer_run_cnt -= dec; - if (timer_run_cnt > 0) + timer_local.run_cnt -= dec; + if (timer_local.run_cnt > 0) return; - timer_run_cnt = poll_interval; + timer_local.run_cnt = poll_interval; } now = odp_time_global(); if (poll_interval > 1) { - odp_time_t period = odp_time_diff(now, last_timer_run); + odp_time_t period = odp_time_diff(now, timer_local.last_run); if (odp_time_cmp(period, timer_global->poll_interval_time) < 0) return; - last_timer_run = now; + timer_local.last_run = now; } /* Check the timer pools. */ @@ -1563,6 +1557,9 @@ int _odp_timer_term_global(void) int _odp_timer_init_local(void) { + timer_local.last_run = odp_time_global_from_ns(0); + timer_local.run_cnt = 1; + return 0; } commit d2a9548017edaceec8cc682183690592c4c483b2 Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Wed Oct 30 11:24:56 2019 +0200 linux-gen: timer: add inline_thread_type config file option Added option to control which thread types process (non-private) inline timer pools. Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> diff --git a/config/odp-linux-generic.conf b/config/odp-linux-generic.conf index 4e24ef82d..b35089944 100644 --- a/config/odp-linux-generic.conf +++ b/config/odp-linux-generic.conf @@ -16,7 +16,7 @@ # Mandatory fields odp_implementation = "linux-generic" -config_file_version = "0.1.9" +config_file_version = "0.1.10" # Shared memory options shm: { @@ -148,18 +148,21 @@ timer: { # Use inline timer implementation # # By default, timer processing is done in background threads (thread per - # timer pool). With inline implementation timers are processed on worker - # cores instead. When using inline timers the application has to call - # odp_schedule() or odp_queue_deq() to actuate timer processing. + # timer pool). With inline implementation timers are processed by ODP + # application threads instead. When using inline timers the application + # has to call odp_schedule() or odp_queue_deq() regularly to actuate + # timer processing. # - # Set to 1 to enable + # 0: Use POSIX timer and background threads to process timers + # 1: Use inline timer implementation and application threads to process + # timers inline = 0 # Inline timer poll interval # # When set to 1 inline timers are polled during every schedule round. # Increasing the value reduces timer processing overhead while - # decreasing accuracy. Ignored when inline timer is not enabled. + # decreasing accuracy. Ignored when inline timer is not used. inline_poll_interval = 10 # Inline timer poll interval in nanoseconds @@ -168,6 +171,18 @@ timer: { # inline timer polling rate in nanoseconds. By default, this defines the # maximum rate a thread may poll timers. If a timer pool is created with # a higher resolution than this, the polling rate is increased - # accordingly. + # accordingly. Ignored when inline timer is not used. inline_poll_interval_nsec = 500000 + + # Inline timer use of threads + # + # Select which thread types process non-private timer pools in inline + # timer implementation. Thread type does not affect private timer + # pool procesessing, those are always processed by the thread which + # created the pool. Ignored when inline timer is not used. + # + # 0: Both control and worker threads process non-private timer pools + # 1: Only worker threads process non-private timer pools + # 2: Only control threads process non-private timer pools + inline_thread_type = 0 } diff --git a/platform/linux-generic/odp_timer.c b/platform/linux-generic/odp_timer.c index e21b29b0c..4af31bcc1 100644 --- a/platform/linux-generic/odp_timer.c +++ b/platform/linux-generic/odp_timer.c @@ -226,6 +226,7 @@ typedef struct timer_global_t { odp_bool_t use_inline_timers; int poll_interval; int highest_tp_idx; + uint8_t thread_type; } timer_global_t; @@ -1511,35 +1512,43 @@ int _odp_timer_init_global(const odp_init_t *params) conf_str = "timer.inline"; if (!_odp_libconfig_lookup_int(conf_str, &val)) { ODP_ERR("Config option '%s' not found.\n", conf_str); - odp_shm_free(shm); - return -1; + goto error; } timer_global->use_inline_timers = val; conf_str = "timer.inline_poll_interval"; if (!_odp_libconfig_lookup_int(conf_str, &val)) { ODP_ERR("Config option '%s' not found.\n", conf_str); - odp_shm_free(shm); - return -1; + goto error; } timer_global->poll_interval = val; conf_str = "timer.inline_poll_interval_nsec"; if (!_odp_libconfig_lookup_int(conf_str, &val)) { ODP_ERR("Config option '%s' not found.\n", conf_str); - odp_shm_free(shm); - return -1; + goto error; } timer_global->poll_interval_nsec = val; timer_global->poll_interval_time = odp_time_global_from_ns(timer_global->poll_interval_nsec); + conf_str = "timer.inline_thread_type"; + if (!_odp_libconfig_lookup_int(conf_str, &val)) { + ODP_ERR("Config option '%s' not found.\n", conf_str); + goto error; + } + timer_global->thread_type = val; + if (!timer_global->use_inline_timers) { timer_res_init(); block_sigalarm(); } return 0; + +error: + odp_shm_free(shm); + return -1; } int _odp_timer_term_global(void) diff --git a/platform/linux-generic/test/inline-timer.conf b/platform/linux-generic/test/inline-timer.conf index 435f51e6d..362ecd559 100644 --- a/platform/linux-generic/test/inline-timer.conf +++ b/platform/linux-generic/test/inline-timer.conf @@ -1,6 +1,6 @@ # Mandatory fields odp_implementation = "linux-generic" -config_file_version = "0.1.9" +config_file_version = "0.1.10" timer: { # Enable inline timer implementation diff --git a/platform/linux-generic/test/process-mode.conf b/platform/linux-generic/test/process-mode.conf index 708b9b4a8..6fb08aeaf 100644 --- a/platform/linux-generic/test/process-mode.conf +++ b/platform/linux-generic/test/process-mode.conf @@ -1,6 +1,6 @@ # Mandatory fields odp_implementation = "linux-generic" -config_file_version = "0.1.9" +config_file_version = "0.1.10" # Shared memory options shm: { commit 68e1cd81665c3b8deede1974d653593af85ec183 Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Tue Oct 29 16:46:05 2019 +0200 linux-gen: timer: coding style fixes Fix most issues reported by the current checkpatch version. Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> diff --git a/platform/linux-generic/odp_timer.c b/platform/linux-generic/odp_timer.c index 1ca61e179..e21b29b0c 100644 --- a/platform/linux-generic/odp_timer.c +++ b/platform/linux-generic/odp_timer.c @@ -87,6 +87,7 @@ static odp_timeout_hdr_t *timeout_hdr_from_buf(odp_buffer_t buf) static odp_timeout_hdr_t *timeout_hdr(odp_timeout_t tmo) { odp_buffer_t buf = odp_buffer_from_event(odp_timeout_to_event(tmo)); + return timeout_hdr_from_buf(buf); } @@ -243,8 +244,10 @@ static inline odp_timer_pool_t timer_pool_to_hdl(timer_pool_t *tp) static inline timer_pool_t *handle_to_tp(odp_timer_t hdl) { uint32_t tp_idx = _odp_typeval(hdl) >> INDEX_BITS; + if (odp_likely(tp_idx < MAX_TIMER_POOLS)) { timer_pool_t *tp = timer_global->timer_pool[tp_idx]; + if (odp_likely(tp != NULL)) return timer_global->timer_pool[tp_idx]; } @@ -255,6 +258,7 @@ static inline uint32_t handle_to_idx(odp_timer_t hdl, timer_pool_t *tp) { uint32_t idx = (_odp_typeval(hdl) & ((1U << INDEX_BITS) - 1U)) - 1; + __builtin_prefetch(&tp->tick_buf[idx], 0, 0); if (odp_likely(idx < odp_atomic_load_u32(&tp->high_wm))) return idx; @@ -471,6 +475,7 @@ static inline odp_timer_t timer_alloc(timer_pool_t *tp, void *user_ptr) { odp_timer_t hdl; + odp_spinlock_lock(&tp->lock); if (odp_likely(tp->num_alloc < tp->param.num_timers)) { tp->num_alloc++; @@ -478,6 +483,7 @@ static inline odp_timer_t timer_alloc(timer_pool_t *tp, ODP_ASSERT(tp->first_free != tp->param.num_timers); uint32_t idx = tp->first_free; _odp_timer_t *tim = &tp->timers[idx]; + tp->first_free = get_next_free(tim); /* Initialize timer */ timer_init(tim, &tp->tick_buf[idx], queue, user_ptr); @@ -532,10 +538,8 @@ static inline odp_buffer_t timer_free(timer_pool_t *tp, uint32_t idx) * expire/reset/cancel timer *****************************************************************************/ -static bool timer_reset(uint32_t idx, - uint64_t abs_tck, - odp_buffer_t *tmo_buf, - timer_pool_t *tp) +static bool timer_reset(uint32_t idx, uint64_t abs_tck, odp_buffer_t *tmo_buf, + timer_pool_t *tp) { bool success = true; tick_buf_t *tb = &tp->tick_buf[idx]; @@ -582,7 +586,7 @@ static bool timer_reset(uint32_t idx, /* Swap in new expiration tick, get back old tick which * will indicate active/inactive timer state */ old = _odp_atomic_u64_xchg_mm(&tb->exp_tck, abs_tck, - _ODP_MEMMODEL_RLX); + _ODP_MEMMODEL_RLX); if ((old & TMO_INACTIVE) != 0) { /* Timer was inactive (cancelled or expired), * we can't reset a timer without a timeout buffer. @@ -747,12 +751,11 @@ static odp_buffer_t timer_cancel(timer_pool_t *tp, /* Atomic CAS will fail if we experienced torn reads, * retry update sequence until CAS succeeds */ - } while (!_odp_atomic_u128_cmp_xchg_mm( - (_odp_atomic_u128_t *)tb, - (_uint128_t *)&old, - (_uint128_t *)&new, - _ODP_MEMMODEL_RLS, - _ODP_MEMMODEL_RLX)); + } while (!_odp_atomic_u128_cmp_xchg_mm((_odp_atomic_u128_t *)tb, + (_uint128_t *)&old, + (_uint128_t *)&new, + _ODP_MEMMODEL_RLS, + _ODP_MEMMODEL_RLX)); old_buf = old.tmo_buf; #else /* Take a related lock */ @@ -881,6 +884,7 @@ static inline void timer_pool_scan(timer_pool_t *tp, uint64_t tick) __builtin_prefetch(&array[i + 32], 0, 0); /* Non-atomic read for speed */ uint64_t exp_tck = array[i].exp_tck.v; + if (odp_unlikely(exp_tck <= tick)) { /* Attempt to expire timer */ timer_expire(tp, i, tick); @@ -1142,7 +1146,7 @@ static void itimer_init(timer_pool_t *tp) int ret; ODP_DBG("Creating POSIX timer for timer pool %s, period %" - PRIu64" ns\n", tp->name, tp->param.res_ns); + PRIu64 " ns\n", tp->name, tp->param.res_ns); res = tp->param.res_ns; sec = res / ODP_TIME_SEC_IN_NS; @@ -1341,6 +1345,7 @@ odp_event_t odp_timer_free(odp_timer_t hdl) timer_pool_t *tp = handle_to_tp(hdl); uint32_t idx = handle_to_idx(hdl, tp); odp_buffer_t old_buf = timer_free(tp, idx); + return odp_buffer_to_event(old_buf); } @@ -1387,6 +1392,7 @@ int odp_timer_cancel(odp_timer_t hdl, odp_event_t *tmo_ev) uint32_t idx = handle_to_idx(hdl, tp); /* Set the expiration tick of the timer to TMO_INACTIVE */ odp_buffer_t old_buf = timer_cancel(tp, idx); + if (old_buf != ODP_BUFFER_INVALID) { *tmo_ev = odp_buffer_to_event(old_buf); return 0; /* Active timer cancelled, timeout returned */ @@ -1453,6 +1459,7 @@ void *odp_timeout_user_ptr(odp_timeout_t tmo) odp_timeout_t odp_timeout_alloc(odp_pool_t pool) { odp_buffer_t buf = odp_buffer_alloc(pool); + if (odp_unlikely(buf == ODP_BUFFER_INVALID)) return ODP_TIMEOUT_INVALID; return odp_timeout_from_event(odp_buffer_to_event(buf)); @@ -1461,6 +1468,7 @@ odp_timeout_t odp_timeout_alloc(odp_pool_t pool) void odp_timeout_free(odp_timeout_t tmo) { odp_event_t ev = odp_timeout_to_event(tmo); + odp_buffer_free(odp_buffer_from_event(ev)); } @@ -1494,6 +1502,7 @@ int _odp_timer_init_global(const odp_init_t *params) #ifndef ODP_ATOMIC_U128 uint32_t i; + for (i = 0; i < NUM_LOCKS; i++) _odp_atomic_flag_clear(&timer_global->locks[i]); #else ----------------------------------------------------------------------- Summary of changes: config/odp-linux-generic.conf | 29 ++- platform/linux-generic/odp_timer.c | 246 ++++++++++++++++---------- platform/linux-generic/test/inline-timer.conf | 2 +- platform/linux-generic/test/process-mode.conf | 2 +- test/performance/odp_timer_perf.c | 73 ++++++-- test/validation/api/scheduler/scheduler.c | 231 ++++++++++++++++++++++++ test/validation/api/timer/timer.c | 24 ++- 7 files changed, 479 insertions(+), 128 deletions(-) hooks/post-receive --

6 years, 10 months

[COMMIT] branch master updated. v1.21.0.0-309-gb921639a9

by git＠git-us.linaro.org

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via b921639a9df6a0300ca1c50b5853a37241806366 (commit) via 6f367faae189144beeaafcd9850065b3f1b1098e (commit) via 4afa7da09edf07126573940fffed0784ef3080ff (commit) via 5db9a7a1c59d4eda436d896dce41a2c7aa8f79c4 (commit) via b89a5b77e92060c9343ab162526d05a8ee97f8b4 (commit) via fc07575511c3a5b79ccae80ca278b3daad4f101d (commit) via ff99ba52f99a608dc1a41f4d8a86ab5c2f8491ca (commit) via 426ee8c61835c8faa5df0873ecc4ec7c8667c691 (commit) via 559aec0db739c624652facd3b337d0430cba8c61 (commit) via 720d655c9161007b561bf033a4fbe0b31ab7fb6a (commit) from 1a8b32c758894b0d0a71ebbd86efb77fd9eb334c (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit b921639a9df6a0300ca1c50b5853a37241806366 Author: Matias Elo <matias.elo(a)nokia.com> Date: Mon Oct 28 17:39:52 2019 +0200 linux-gen: ipc: remove ipc_data_offset from buffer header Instead of storing IPC packet data offset in odp_buffer_hdr_t, calculate the offset on receive using odp_packet_hdr_t.seg_data pointer and remote pool base address. Signed-off-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen(a)nokia.com> diff --git a/platform/linux-generic/include/odp_buffer_internal.h b/platform/linux-generic/include/odp_buffer_internal.h index 1c52b028f..6c6ec970d 100644 --- a/platform/linux-generic/include/odp_buffer_internal.h +++ b/platform/linux-generic/include/odp_buffer_internal.h @@ -68,10 +68,6 @@ struct ODP_ALIGNED_CACHE odp_buffer_hdr_t { /* User area pointer */ void *uarea_addr; - /* ipc mapped process can not walk over pointers, - * offset has to be used */ - uint64_t ipc_data_offset; - /* Combined pool and buffer index */ buffer_index_t index; diff --git a/platform/linux-generic/pktio/ipc.c b/platform/linux-generic/pktio/ipc.c index 7e3f95fb6..7575eb052 100644 --- a/platform/linux-generic/pktio/ipc.c +++ b/platform/linux-generic/pktio/ipc.c @@ -35,8 +35,9 @@ */ struct pktio_info { struct { + /* Pool base address */ + void *base_addr; /* number of buffer*/ - int num; char pool_name[ODP_POOL_NAME_LEN]; /* 1 if master finished creation of all shared objects */ int init_done; @@ -46,6 +47,7 @@ struct pktio_info { uint32_t ring_mask; } master; struct { + /* Pool base address */ void *base_addr; char pool_name[ODP_POOL_NAME_LEN]; /* pid of the slave process written to shm and @@ -78,8 +80,10 @@ typedef struct { /* local cache to keep packet order right */ ring_ptr_t *cache; } rx; /* slave */ - void *pool_base; /**< Remote pool base addr */ - void *pool_mdata_base; /**< Remote pool mdata base addr */ + /* Remote pool mdata base addr */ + void *pool_mdata_base; + /* Remote pool base address for offset calculation */ + void *remote_base_addr; odp_pool_t pool; /**< Pool of main process */ enum { PKTIO_TYPE_IPC_MASTER = 0, /**< Master is the process which @@ -212,7 +216,7 @@ static int _ipc_master_start(pktio_entry_t *pktio_entry) } pktio_ipc->remote_pool_shm = shm; - pktio_ipc->pool_base = odp_shm_addr(shm); + pktio_ipc->remote_base_addr = pinfo->slave.base_addr; pktio_ipc->pool_mdata_base = (char *)odp_shm_addr(shm); odp_atomic_store_u32(&pktio_ipc->ready, 1); @@ -328,6 +332,7 @@ static int _ipc_init_master(pktio_entry_t *pktio_entry, /* Export ring info for the slave process to use */ pinfo->master.ring_size = ring_size; pinfo->master.ring_mask = ring_mask; + pinfo->master.base_addr = odp_shm_addr(pool->shm); pinfo->slave.base_addr = 0; pinfo->slave.pid = 0; @@ -362,7 +367,7 @@ static void _ipc_export_pool(struct pktio_info *pinfo, snprintf(pinfo->slave.pool_name, ODP_POOL_NAME_LEN, "%s", _ipc_odp_buffer_pool_shm_name(pool_hdl)); pinfo->slave.pid = odp_global_ro.main_pid; - pinfo->slave.base_addr = pool->base_addr; + pinfo->slave.base_addr = odp_shm_addr(pool->shm); } static odp_shm_t _ipc_map_remote_pool(const char *name, int pid) @@ -494,6 +499,7 @@ static int _ipc_slave_start(pktio_entry_t *pktio_entry) pid); pktio_ipc->remote_pool_shm = shm; pktio_ipc->pool_mdata_base = (char *)odp_shm_addr(shm); + pktio_ipc->remote_base_addr = pinfo->master.base_addr; _ipc_export_pool(pinfo, pktio_ipc->pool); @@ -674,7 +680,8 @@ static int ipc_pktio_recv_lockless(pktio_entry_t *pktio_entry, if (odp_unlikely(pool == ODP_POOL_INVALID)) ODP_ABORT("invalid pool"); - data_pool_off = phdr->buf_hdr.ipc_data_offset; + data_pool_off = (uint8_t *)phdr->seg_data - + (uint8_t *)pktio_ipc->remote_base_addr; pkt = odp_packet_alloc(pool, phdr->frame_len); if (odp_unlikely(pkt == ODP_PACKET_INVALID)) { @@ -803,7 +810,6 @@ static int ipc_pktio_send_lockless(pktio_entry_t *pktio_entry, /* Set offset to phdr for outgoing packets */ for (i = 0; i < num; i++) { - uint64_t data_pool_off; odp_packet_t pkt = pkt_table_mapped[i]; odp_packet_hdr_t *pkt_hdr = packet_hdr(pkt); odp_pool_t pool_hdl = odp_packet_pool(pkt); @@ -811,17 +817,15 @@ static int ipc_pktio_send_lockless(pktio_entry_t *pktio_entry, offsets[i] = (uint8_t *)pkt_hdr - (uint8_t *)odp_shm_addr(pool->shm); - data_pool_off = (uint8_t *)pkt_hdr->seg_data - - (uint8_t *)odp_shm_addr(pool->shm); /* compile all function code even if ipc disabled with config */ - pkt_hdr->buf_hdr.ipc_data_offset = data_pool_off; IPC_ODP_DBG("%d/%d send packet %llx, pool %llx," - "phdr = %p, offset %x sendoff %x, addr %llx iaddr %llx\n", + "phdr = %p, offset %x, sendoff %x, addr %llx iaddr %llx\n", i, num, odp_packet_to_u64(pkt), odp_pool_to_u64(pool_hdl), - pkt_hdr, pkt_hdr->buf_hdr.ipc_data_offset, - offsets[i], odp_shm_addr(pool->shm), + pkt_hdr, (uint8_t *)pkt_hdr->seg_data - + (uint8_t *)odp_shm_addr(pool->shm), offsets[i], + odp_shm_addr(pool->shm), odp_shm_addr(ipc_pool->shm)); } commit 6f367faae189144beeaafcd9850065b3f1b1098e Author: Matias Elo <matias.elo(a)nokia.com> Date: Mon Oct 28 15:01:47 2019 +0200 linux-gen: ipc: add IPC_BURST_SIZE define for IPC internal burst size Signed-off-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen(a)nokia.com> diff --git a/platform/linux-generic/pktio/ipc.c b/platform/linux-generic/pktio/ipc.c index 572d02e74..7e3f95fb6 100644 --- a/platform/linux-generic/pktio/ipc.c +++ b/platform/linux-generic/pktio/ipc.c @@ -27,6 +27,9 @@ ODP_DBG(fmt, ##__VA_ARGS__);\ } while (0) +/* Burst size for IPC free operations */ +#define IPC_BURST_SIZE 32 + /* that struct is exported to shared memory, so that processes can find * each other. */ @@ -237,7 +240,12 @@ static int _ipc_init_master(pktio_entry_t *pktio_entry, /* Ring must be able to store all packets in the pool */ ring_size = ROUNDUP_POWER2_U32(pool->num + 1); + + /* Ring size has to larger than burst size */ + if (ring_size <= IPC_BURST_SIZE) + ring_size = ROUNDUP_POWER2_U32(IPC_BURST_SIZE + 1); ring_mask = ring_size - 1; + pktio_ipc->ring_size = ring_size; pktio_ipc->ring_mask = ring_mask; @@ -579,9 +587,7 @@ static void _ipc_free_ring_packets(pktio_entry_t *pktio_entry, ring_ptr_t *r, uint32_t r_mask) { pkt_ipc_t *pktio_ipc = pkt_priv(pktio_entry); - uint32_t ring_size = pktio_ipc->ring_size; - uint32_t ring_mask = pktio_ipc->ring_mask; - uintptr_t offsets[ring_size]; + uintptr_t offsets[IPC_BURST_SIZE]; int ret; void **rbuf_p; int i; @@ -597,7 +603,7 @@ static void _ipc_free_ring_packets(pktio_entry_t *pktio_entry, ring_ptr_t *r, rbuf_p = (void *)&offsets; while (1) { - ret = ring_ptr_deq_multi(r, r_mask, rbuf_p, ring_mask); + ret = ring_ptr_deq_multi(r, r_mask, rbuf_p, IPC_BURST_SIZE); if (ret <= 0) break; for (i = 0; i < ret; i++) { @@ -616,13 +622,12 @@ static int ipc_pktio_recv_lockless(pktio_entry_t *pktio_entry, odp_packet_t pkt_table[], int len) { pkt_ipc_t *pktio_ipc = pkt_priv(pktio_entry); - uint32_t ring_size = pktio_ipc->ring_size; uint32_t ring_mask = pktio_ipc->ring_mask; int pkts = 0; int i; ring_ptr_t *r; ring_ptr_t *r_p; - uintptr_t offsets[ring_size]; + uintptr_t offsets[len]; void **ipcbufs_p = (void *)&offsets[0]; uint32_t ready; commit 4afa7da09edf07126573940fffed0784ef3080ff Author: Matias Elo <matias.elo(a)nokia.com> Date: Mon Oct 28 10:52:38 2019 +0200 linux-gen: ipc: clean excessive pkt_priv() usage Signed-off-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen(a)nokia.com> diff --git a/platform/linux-generic/pktio/ipc.c b/platform/linux-generic/pktio/ipc.c index 065f70cc7..572d02e74 100644 --- a/platform/linux-generic/pktio/ipc.c +++ b/platform/linux-generic/pktio/ipc.c @@ -193,7 +193,8 @@ static const char *_ipc_odp_buffer_pool_shm_name(odp_pool_t pool_hdl) static int _ipc_master_start(pktio_entry_t *pktio_entry) { - struct pktio_info *pinfo = pkt_priv(pktio_entry)->pinfo; + pkt_ipc_t *pktio_ipc = pkt_priv(pktio_entry); + struct pktio_info *pinfo = pktio_ipc->pinfo; odp_shm_t shm; if (pinfo->slave.init_done == 0) @@ -207,11 +208,11 @@ static int _ipc_master_start(pktio_entry_t *pktio_entry) return -1; } - pkt_priv(pktio_entry)->remote_pool_shm = shm; - pkt_priv(pktio_entry)->pool_base = odp_shm_addr(shm); - pkt_priv(pktio_entry)->pool_mdata_base = (char *)odp_shm_addr(shm); + pktio_ipc->remote_pool_shm = shm; + pktio_ipc->pool_base = odp_shm_addr(shm); + pktio_ipc->pool_mdata_base = (char *)odp_shm_addr(shm); - odp_atomic_store_u32(&pkt_priv(pktio_entry)->ready, 1); + odp_atomic_store_u32(&pktio_ipc->ready, 1); IPC_ODP_DBG("%s started.\n", pktio_entry->s.name); return 0; @@ -221,6 +222,7 @@ static int _ipc_init_master(pktio_entry_t *pktio_entry, const char *dev, odp_pool_t pool_hdl) { + pkt_ipc_t *pktio_ipc = pkt_priv(pktio_entry); char ipc_shm_name[ODP_POOL_NAME_LEN + sizeof("_m_prod")]; struct pktio_info *pinfo; const char *pool_name; @@ -236,17 +238,16 @@ static int _ipc_init_master(pktio_entry_t *pktio_entry, /* Ring must be able to store all packets in the pool */ ring_size = ROUNDUP_POWER2_U32(pool->num + 1); ring_mask = ring_size - 1; - pkt_priv(pktio_entry)->ring_size = ring_size; - pkt_priv(pktio_entry)->ring_mask = ring_mask; + pktio_ipc->ring_size = ring_size; + pktio_ipc->ring_mask = ring_mask; if (strlen(dev) > (ODP_POOL_NAME_LEN - sizeof("_m_prod"))) { ODP_ERR("too big ipc name\n"); return -1; } - pkt_priv(pktio_entry)->rx.cache = _ring_create("ipc_rx_cache", - ring_size, 0); - if (!pkt_priv(pktio_entry)->rx.cache) { + pktio_ipc->rx.cache = _ring_create("ipc_rx_cache", ring_size, 0); + if (!pktio_ipc->rx.cache) { ODP_ERR("pid %d unable to create ipc rx cache\n", getpid()); return -1; } @@ -255,66 +256,58 @@ static int _ipc_init_master(pktio_entry_t *pktio_entry, * to be processed packets ring. */ snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_m_prod", dev); - pkt_priv(pktio_entry)->tx.send = _ring_create(ipc_shm_name, ring_size, - ODP_SHM_PROC | - ODP_SHM_EXPORT); - if (!pkt_priv(pktio_entry)->tx.send) { + pktio_ipc->tx.send = _ring_create(ipc_shm_name, ring_size, + ODP_SHM_PROC | ODP_SHM_EXPORT); + if (!pktio_ipc->tx.send) { ODP_ERR("pid %d unable to create ipc ring %s name\n", getpid(), ipc_shm_name); return -1; } ODP_DBG("Created IPC ring: %s, count %d, free %d\n", - ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->tx.send, - ring_mask), - _ring_free_count(pkt_priv(pktio_entry)->tx.send, ring_mask)); + ipc_shm_name, _ring_count(pktio_ipc->tx.send, ring_mask), + _ring_free_count(pktio_ipc->tx.send, ring_mask)); /* generate name in shm like ipc_pktio_p for * already processed packets */ snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_m_cons", dev); - pkt_priv(pktio_entry)->tx.free = _ring_create(ipc_shm_name, ring_size, - ODP_SHM_PROC | - ODP_SHM_EXPORT); - if (!pkt_priv(pktio_entry)->tx.free) { + pktio_ipc->tx.free = _ring_create(ipc_shm_name, ring_size, + ODP_SHM_PROC | ODP_SHM_EXPORT); + if (!pktio_ipc->tx.free) { ODP_ERR("pid %d unable to create ipc ring %s name\n", getpid(), ipc_shm_name); goto free_m_prod; } ODP_DBG("Created IPC ring: %s, count %d, free %d\n", - ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->tx.free, - ring_mask), - _ring_free_count(pkt_priv(pktio_entry)->tx.free, ring_mask)); + ipc_shm_name, _ring_count(pktio_ipc->tx.free, ring_mask), + _ring_free_count(pktio_ipc->tx.free, ring_mask)); snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_s_prod", dev); - pkt_priv(pktio_entry)->rx.recv = _ring_create(ipc_shm_name, ring_size, - ODP_SHM_PROC | - ODP_SHM_EXPORT); - if (!pkt_priv(pktio_entry)->rx.recv) { + pktio_ipc->rx.recv = _ring_create(ipc_shm_name, ring_size, + ODP_SHM_PROC | ODP_SHM_EXPORT); + if (!pktio_ipc->rx.recv) { ODP_ERR("pid %d unable to create ipc ring %s name\n", getpid(), ipc_shm_name); goto free_m_cons; } ODP_DBG("Created IPC ring: %s, count %d, free %d\n", - ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->rx.recv, - ring_mask), - _ring_free_count(pkt_priv(pktio_entry)->rx.recv, ring_mask)); + ipc_shm_name, _ring_count(pktio_ipc->rx.recv, ring_mask), + _ring_free_count(pktio_ipc->rx.recv, ring_mask)); snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_s_cons", dev); - pkt_priv(pktio_entry)->rx.free = _ring_create(ipc_shm_name, ring_size, - ODP_SHM_PROC | - ODP_SHM_EXPORT); - if (!pkt_priv(pktio_entry)->rx.free) { + pktio_ipc->rx.free = _ring_create(ipc_shm_name, ring_size, + ODP_SHM_PROC | ODP_SHM_EXPORT); + if (!pktio_ipc->rx.free) { ODP_ERR("pid %d unable to create ipc ring %s name\n", getpid(), ipc_shm_name); goto free_s_prod; } ODP_DBG("Created IPC ring: %s, count %d, free %d\n", - ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->rx.free, - ring_mask), - _ring_free_count(pkt_priv(pktio_entry)->rx.free, ring_mask)); + ipc_shm_name, _ring_count(pktio_ipc->rx.free, ring_mask), + _ring_free_count(pktio_ipc->rx.free, ring_mask)); /* Set up pool name for remote info */ - pinfo = pkt_priv(pktio_entry)->pinfo; + pinfo = pktio_ipc->pinfo; pool_name = _ipc_odp_buffer_pool_shm_name(pool_hdl); if (strlen(pool_name) > ODP_POOL_NAME_LEN) { ODP_ERR("pid %d ipc pool name %s is too big %d\n", @@ -332,7 +325,7 @@ static int _ipc_init_master(pktio_entry_t *pktio_entry, pinfo->slave.pid = 0; pinfo->slave.init_done = 0; - pkt_priv(pktio_entry)->pool = pool_hdl; + pktio_ipc->pool = pool_hdl; ODP_DBG("Pre init... DONE.\n"); pinfo->master.init_done = 1; @@ -426,13 +419,14 @@ static int _ipc_init_slave(const char *dev, pktio_entry_t *pktio_entry, static int _ipc_slave_start(pktio_entry_t *pktio_entry) { + pkt_ipc_t *pktio_ipc = pkt_priv(pktio_entry); char ipc_shm_name[ODP_POOL_NAME_LEN + sizeof("_slave_r")]; struct pktio_info *pinfo; odp_shm_t shm; char tail[ODP_POOL_NAME_LEN]; char dev[ODP_POOL_NAME_LEN]; int pid; - uint32_t ring_mask = pkt_priv(pktio_entry)->ring_mask; + uint32_t ring_mask = pktio_ipc->ring_mask; if (sscanf(pktio_entry->s.name, "ipc:%d:%s", &pid, tail) != 2) { ODP_ERR("wrong pktio name\n"); @@ -442,68 +436,60 @@ static int _ipc_slave_start(pktio_entry_t *pktio_entry) sprintf(dev, "ipc:%s", tail); snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_m_prod", dev); - pkt_priv(pktio_entry)->rx.recv = _ipc_shm_map(ipc_shm_name, pid); - if (!pkt_priv(pktio_entry)->rx.recv) { + pktio_ipc->rx.recv = _ipc_shm_map(ipc_shm_name, pid); + if (!pktio_ipc->rx.recv) { ODP_DBG("pid %d unable to find ipc ring %s name\n", getpid(), dev); sleep(1); return -1; } ODP_DBG("Connected IPC ring: %s, count %d, free %d\n", - ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->rx.recv, - ring_mask), - _ring_free_count(pkt_priv(pktio_entry)->rx.recv, - ring_mask)); + ipc_shm_name, _ring_count(pktio_ipc->rx.recv, ring_mask), + _ring_free_count(pktio_ipc->rx.recv, ring_mask)); snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_m_cons", dev); - pkt_priv(pktio_entry)->rx.free = _ipc_shm_map(ipc_shm_name, pid); - if (!pkt_priv(pktio_entry)->rx.free) { + pktio_ipc->rx.free = _ipc_shm_map(ipc_shm_name, pid); + if (!pktio_ipc->rx.free) { ODP_ERR("pid %d unable to find ipc ring %s name\n", getpid(), dev); goto free_m_prod; } ODP_DBG("Connected IPC ring: %s, count %d, free %d\n", - ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->rx.free, - ring_mask), - _ring_free_count(pkt_priv(pktio_entry)->rx.free, - ring_mask)); + ipc_shm_name, _ring_count(pktio_ipc->rx.free, ring_mask), + _ring_free_count(pktio_ipc->rx.free, ring_mask)); snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_s_prod", dev); - pkt_priv(pktio_entry)->tx.send = _ipc_shm_map(ipc_shm_name, pid); - if (!pkt_priv(pktio_entry)->tx.send) { + pktio_ipc->tx.send = _ipc_shm_map(ipc_shm_name, pid); + if (!pktio_ipc->tx.send) { ODP_ERR("pid %d unable to find ipc ring %s name\n", getpid(), dev); goto free_m_cons; } ODP_DBG("Connected IPC ring: %s, count %d, free %d\n", - ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->tx.send, - ring_mask), - _ring_free_count(pkt_priv(pktio_entry)->tx.send, - ring_mask)); + ipc_shm_name, _ring_count(pktio_ipc->tx.send, ring_mask), + _ring_free_count(pktio_ipc->tx.send, ring_mask)); snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_s_cons", dev); - pkt_priv(pktio_entry)->tx.free = _ipc_shm_map(ipc_shm_name, pid); - if (!pkt_priv(pktio_entry)->tx.free) { + pktio_ipc->tx.free = _ipc_shm_map(ipc_shm_name, pid); + if (!pktio_ipc->tx.free) { ODP_ERR("pid %d unable to find ipc ring %s name\n", getpid(), dev); goto free_s_prod; } ODP_DBG("Connected IPC ring: %s, count %d, free %d\n", - ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->tx.free, - ring_mask), - _ring_free_count(pkt_priv(pktio_entry)->tx.free, - ring_mask)); + ipc_shm_name, _ring_count(pktio_ipc->tx.free, ring_mask), + _ring_free_count(pktio_ipc->tx.free, ring_mask)); /* Get info about remote pool */ - pinfo = pkt_priv(pktio_entry)->pinfo; + pinfo = pktio_ipc->pinfo; shm = _ipc_map_remote_pool(pinfo->master.pool_name, pid); - pkt_priv(pktio_entry)->remote_pool_shm = shm; - pkt_priv(pktio_entry)->pool_mdata_base = (char *)odp_shm_addr(shm); + pktio_ipc->remote_pool_shm = shm; + pktio_ipc->pool_mdata_base = (char *)odp_shm_addr(shm); - _ipc_export_pool(pinfo, pkt_priv(pktio_entry)->pool); + _ipc_export_pool(pinfo, pktio_ipc->pool); - odp_atomic_store_u32(&pkt_priv(pktio_entry)->ready, 1); + odp_atomic_store_u32(&pktio_ipc->ready, 1); pinfo->slave.init_done = 1; ODP_DBG("%s started.\n", pktio_entry->s.name); @@ -871,14 +857,15 @@ static int ipc_mac_addr_get(pktio_entry_t *pktio_entry ODP_UNUSED, static int ipc_start(pktio_entry_t *pktio_entry) { - uint32_t ready = odp_atomic_load_u32(&pkt_priv(pktio_entry)->ready); + pkt_ipc_t *pktio_ipc = pkt_priv(pktio_entry); + uint32_t ready = odp_atomic_load_u32(&pktio_ipc->ready); if (ready) { ODP_ABORT("%s Already started\n", pktio_entry->s.name); return -1; } - if (pkt_priv(pktio_entry)->type == PKTIO_TYPE_IPC_MASTER) + if (pktio_ipc->type == PKTIO_TYPE_IPC_MASTER) return _ipc_master_start(pktio_entry); else return _ipc_slave_start(pktio_entry); @@ -906,6 +893,7 @@ static int ipc_stop(pktio_entry_t *pktio_entry) static int ipc_close(pktio_entry_t *pktio_entry) { + pkt_ipc_t *pktio_ipc = pkt_priv(pktio_entry); char ipc_shm_name[ODP_POOL_NAME_LEN + sizeof("_m_prod")]; char *dev = pktio_entry->s.name; char name[ODP_POOL_NAME_LEN]; @@ -914,7 +902,7 @@ static int ipc_close(pktio_entry_t *pktio_entry) ipc_stop(pktio_entry); - odp_shm_free(pkt_priv(pktio_entry)->remote_pool_shm); + odp_shm_free(pktio_ipc->remote_pool_shm); if (sscanf(dev, "ipc:%d:%s", &pid, tail) == 2) snprintf(name, sizeof(name), "ipc:%s", tail); @@ -922,7 +910,7 @@ static int ipc_close(pktio_entry_t *pktio_entry) snprintf(name, sizeof(name), "%s", dev); /* unlink this pktio info for both master and slave */ - odp_shm_free(pkt_priv(pktio_entry)->pinfo_shm); + odp_shm_free(pktio_ipc->pinfo_shm); /* destroy rings */ snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_s_cons", name); commit 5db9a7a1c59d4eda436d896dce41a2c7aa8f79c4 Author: Matias Elo <matias.elo(a)nokia.com> Date: Mon Oct 28 10:40:42 2019 +0200 linux-gen: ipc: match ring size to used packet pools The ring_ptr_t implementation requires that ring size must be larger than the maximum number of data items that will be stored on it. Adjust ring size according to the used packet pools to guarantee this. Signed-off-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen(a)nokia.com> diff --git a/platform/linux-generic/pktio/ipc.c b/platform/linux-generic/pktio/ipc.c index dec918e95..065f70cc7 100644 --- a/platform/linux-generic/pktio/ipc.c +++ b/platform/linux-generic/pktio/ipc.c @@ -27,11 +27,6 @@ ODP_DBG(fmt, ##__VA_ARGS__);\ } while (0) -/* number of odp buffers in odp ring queue */ -#define PKTIO_IPC_ENTRIES 4096 - -#define PKTIO_IPC_ENTRY_MASK (PKTIO_IPC_ENTRIES - 1) - /* that struct is exported to shared memory, so that processes can find * each other. */ @@ -42,6 +37,10 @@ struct pktio_info { char pool_name[ODP_POOL_NAME_LEN]; /* 1 if master finished creation of all shared objects */ int init_done; + /* IPC ring size */ + uint32_t ring_size; + /* IPC ring mask */ + uint32_t ring_mask; } master; struct { void *base_addr; @@ -87,7 +86,11 @@ typedef struct { } type; /**< define if it's master or slave process */ odp_atomic_u32_t ready; /**< 1 - pktio is ready and can recv/send packet, 0 - not yet ready */ - void *pinfo; + /* Local copy of IPC ring size */ + uint32_t ring_size; + /* Local copy IPC ring mask */ + uint32_t ring_mask; + struct pktio_info *pinfo; odp_shm_t pinfo_shm; odp_shm_t remote_pool_shm; /**< shm of remote pool get with _ipc_map_remote_pool() */ @@ -221,18 +224,38 @@ static int _ipc_init_master(pktio_entry_t *pktio_entry, char ipc_shm_name[ODP_POOL_NAME_LEN + sizeof("_m_prod")]; struct pktio_info *pinfo; const char *pool_name; + pool_t *pool = pool_entry_from_hdl(pool_hdl); + uint32_t ring_size; + uint32_t ring_mask; + + if ((uint64_t)ROUNDUP_POWER2_U32(pool->num + 1) > UINT32_MAX) { + ODP_ERR("Too large packet pool\n"); + return -1; + } + + /* Ring must be able to store all packets in the pool */ + ring_size = ROUNDUP_POWER2_U32(pool->num + 1); + ring_mask = ring_size - 1; + pkt_priv(pktio_entry)->ring_size = ring_size; + pkt_priv(pktio_entry)->ring_mask = ring_mask; if (strlen(dev) > (ODP_POOL_NAME_LEN - sizeof("_m_prod"))) { ODP_ERR("too big ipc name\n"); return -1; } + pkt_priv(pktio_entry)->rx.cache = _ring_create("ipc_rx_cache", + ring_size, 0); + if (!pkt_priv(pktio_entry)->rx.cache) { + ODP_ERR("pid %d unable to create ipc rx cache\n", getpid()); + return -1; + } + /* generate name in shm like ipc_pktio_r for * to be processed packets ring. */ snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_m_prod", dev); - pkt_priv(pktio_entry)->tx.send = _ring_create(ipc_shm_name, - PKTIO_IPC_ENTRIES, + pkt_priv(pktio_entry)->tx.send = _ring_create(ipc_shm_name, ring_size, ODP_SHM_PROC | ODP_SHM_EXPORT); if (!pkt_priv(pktio_entry)->tx.send) { @@ -242,16 +265,14 @@ static int _ipc_init_master(pktio_entry_t *pktio_entry, } ODP_DBG("Created IPC ring: %s, count %d, free %d\n", ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->tx.send, - PKTIO_IPC_ENTRY_MASK), - _ring_free_count(pkt_priv(pktio_entry)->tx.send, - PKTIO_IPC_ENTRY_MASK)); + ring_mask), + _ring_free_count(pkt_priv(pktio_entry)->tx.send, ring_mask)); /* generate name in shm like ipc_pktio_p for * already processed packets */ snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_m_cons", dev); - pkt_priv(pktio_entry)->tx.free = _ring_create(ipc_shm_name, - PKTIO_IPC_ENTRIES, + pkt_priv(pktio_entry)->tx.free = _ring_create(ipc_shm_name, ring_size, ODP_SHM_PROC | ODP_SHM_EXPORT); if (!pkt_priv(pktio_entry)->tx.free) { @@ -261,13 +282,11 @@ static int _ipc_init_master(pktio_entry_t *pktio_entry, } ODP_DBG("Created IPC ring: %s, count %d, free %d\n", ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->tx.free, - PKTIO_IPC_ENTRY_MASK), - _ring_free_count(pkt_priv(pktio_entry)->tx.free, - PKTIO_IPC_ENTRY_MASK)); + ring_mask), + _ring_free_count(pkt_priv(pktio_entry)->tx.free, ring_mask)); snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_s_prod", dev); - pkt_priv(pktio_entry)->rx.recv = _ring_create(ipc_shm_name, - PKTIO_IPC_ENTRIES, + pkt_priv(pktio_entry)->rx.recv = _ring_create(ipc_shm_name, ring_size, ODP_SHM_PROC | ODP_SHM_EXPORT); if (!pkt_priv(pktio_entry)->rx.recv) { @@ -277,13 +296,11 @@ static int _ipc_init_master(pktio_entry_t *pktio_entry, } ODP_DBG("Created IPC ring: %s, count %d, free %d\n", ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->rx.recv, - PKTIO_IPC_ENTRY_MASK), - _ring_free_count(pkt_priv(pktio_entry)->rx.recv, - PKTIO_IPC_ENTRY_MASK)); + ring_mask), + _ring_free_count(pkt_priv(pktio_entry)->rx.recv, ring_mask)); snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_s_cons", dev); - pkt_priv(pktio_entry)->rx.free = _ring_create(ipc_shm_name, - PKTIO_IPC_ENTRIES, + pkt_priv(pktio_entry)->rx.free = _ring_create(ipc_shm_name, ring_size, ODP_SHM_PROC | ODP_SHM_EXPORT); if (!pkt_priv(pktio_entry)->rx.free) { @@ -293,9 +310,8 @@ static int _ipc_init_master(pktio_entry_t *pktio_entry, } ODP_DBG("Created IPC ring: %s, count %d, free %d\n", ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->rx.free, - PKTIO_IPC_ENTRY_MASK), - _ring_free_count(pkt_priv(pktio_entry)->rx.free, - PKTIO_IPC_ENTRY_MASK)); + ring_mask), + _ring_free_count(pkt_priv(pktio_entry)->rx.free, ring_mask)); /* Set up pool name for remote info */ pinfo = pkt_priv(pktio_entry)->pinfo; @@ -307,6 +323,11 @@ static int _ipc_init_master(pktio_entry_t *pktio_entry, } memcpy(pinfo->master.pool_name, pool_name, strlen(pool_name)); + + /* Export ring info for the slave process to use */ + pinfo->master.ring_size = ring_size; + pinfo->master.ring_mask = ring_mask; + pinfo->slave.base_addr = 0; pinfo->slave.pid = 0; pinfo->slave.init_done = 0; @@ -372,14 +393,34 @@ static void *_ipc_shm_map(char *name, int pid) return odp_shm_addr(shm); } -static int _ipc_init_slave(const char *dev, - pktio_entry_t *pktio_entry, - odp_pool_t pool) +static int _ipc_init_slave(const char *dev, pktio_entry_t *pktio_entry, + odp_pool_t pool_hdl) { - if (strlen(dev) > (ODP_POOL_NAME_LEN - sizeof("_slave_r"))) - ODP_ABORT("too big ipc name\n"); + pkt_ipc_t *pktio_ipc = pkt_priv(pktio_entry); + pool_t *pool = pool_entry_from_hdl(pool_hdl); + uint32_t ring_size = pktio_ipc->pinfo->master.ring_size; + + if (strlen(dev) > (ODP_POOL_NAME_LEN - sizeof("_slave_r"))) { + ODP_ERR("Too big ipc name\n"); + return -1; + } + + /* Check that IPC rings are able to store all packets */ + if (pool->num >= ring_size) { + ODP_ERR("Slave process packet pool too large. Master process " + "packet pool has to be larger than slave pool.\n"); + return -1; + } + + pktio_ipc->rx.cache = _ring_create("ipc_rx_cache", ring_size, 0); + if (!pktio_ipc->rx.cache) { + ODP_ERR("Pid %d unable to create ipc rx cache\n", getpid()); + return -1; + } + pktio_ipc->ring_size = ring_size; + pktio_ipc->ring_mask = pktio_ipc->pinfo->master.ring_mask; + pktio_ipc->pool = pool_hdl; - pkt_priv(pktio_entry)->pool = pool; return 0; } @@ -391,6 +432,7 @@ static int _ipc_slave_start(pktio_entry_t *pktio_entry) char tail[ODP_POOL_NAME_LEN]; char dev[ODP_POOL_NAME_LEN]; int pid; + uint32_t ring_mask = pkt_priv(pktio_entry)->ring_mask; if (sscanf(pktio_entry->s.name, "ipc:%d:%s", &pid, tail) != 2) { ODP_ERR("wrong pktio name\n"); @@ -409,9 +451,9 @@ static int _ipc_slave_start(pktio_entry_t *pktio_entry) } ODP_DBG("Connected IPC ring: %s, count %d, free %d\n", ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->rx.recv, - PKTIO_IPC_ENTRY_MASK), + ring_mask), _ring_free_count(pkt_priv(pktio_entry)->rx.recv, - PKTIO_IPC_ENTRY_MASK)); + ring_mask)); snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_m_cons", dev); pkt_priv(pktio_entry)->rx.free = _ipc_shm_map(ipc_shm_name, pid); @@ -422,9 +464,9 @@ static int _ipc_slave_start(pktio_entry_t *pktio_entry) } ODP_DBG("Connected IPC ring: %s, count %d, free %d\n", ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->rx.free, - PKTIO_IPC_ENTRY_MASK), + ring_mask), _ring_free_count(pkt_priv(pktio_entry)->rx.free, - PKTIO_IPC_ENTRY_MASK)); + ring_mask)); snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_s_prod", dev); pkt_priv(pktio_entry)->tx.send = _ipc_shm_map(ipc_shm_name, pid); @@ -435,9 +477,9 @@ static int _ipc_slave_start(pktio_entry_t *pktio_entry) } ODP_DBG("Connected IPC ring: %s, count %d, free %d\n", ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->tx.send, - PKTIO_IPC_ENTRY_MASK), + ring_mask), _ring_free_count(pkt_priv(pktio_entry)->tx.send, - PKTIO_IPC_ENTRY_MASK)); + ring_mask)); snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_s_cons", dev); pkt_priv(pktio_entry)->tx.free = _ipc_shm_map(ipc_shm_name, pid); @@ -448,9 +490,9 @@ static int _ipc_slave_start(pktio_entry_t *pktio_entry) } ODP_DBG("Connected IPC ring: %s, count %d, free %d\n", ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->tx.free, - PKTIO_IPC_ENTRY_MASK), + ring_mask), _ring_free_count(pkt_priv(pktio_entry)->tx.free, - PKTIO_IPC_ENTRY_MASK)); + ring_mask)); /* Get info about remote pool */ pinfo = pkt_priv(pktio_entry)->pinfo; @@ -487,7 +529,8 @@ static int ipc_pktio_open(odp_pktio_t id ODP_UNUSED, const char *dev, odp_pool_t pool) { - int ret = -1; + pkt_ipc_t *pktio_ipc = pkt_priv(pktio_entry); + int ret = 0; int pid; struct pktio_info *pinfo; char name[ODP_POOL_NAME_LEN + sizeof("_info")]; @@ -497,43 +540,35 @@ static int ipc_pktio_open(odp_pktio_t id ODP_UNUSED, if (strncmp(dev, "ipc", 3)) return -1; - odp_atomic_init_u32(&pkt_priv(pktio_entry)->ready, 0); - - pkt_priv(pktio_entry)->rx.cache = _ring_create("ipc_rx_cache", - PKTIO_IPC_ENTRIES, 0); - if (!pkt_priv(pktio_entry)->rx.cache) - return -1; + odp_atomic_init_u32(&pktio_ipc->ready, 0); /* Shared info about remote pktio */ if (sscanf(dev, "ipc:%d:%s", &pid, tail) == 2) { - pkt_priv(pktio_entry)->type = PKTIO_TYPE_IPC_SLAVE; + pktio_ipc->type = PKTIO_TYPE_IPC_SLAVE; snprintf(name, sizeof(name), "ipc:%s_info", tail); IPC_ODP_DBG("lookup for name %s for pid %d\n", name, pid); shm = odp_shm_import(name, pid, name); - if (ODP_SHM_INVALID == shm) { - _ring_destroy("ipc_rx_cache"); + if (ODP_SHM_INVALID == shm) return -1; - } + pinfo = odp_shm_addr(shm); if (!pinfo->master.init_done) { odp_shm_free(shm); - _ring_destroy("ipc_rx_cache"); return -1; } - pkt_priv(pktio_entry)->pinfo = pinfo; - pkt_priv(pktio_entry)->pinfo_shm = shm; + pktio_ipc->pinfo = pinfo; + pktio_ipc->pinfo_shm = shm; ODP_DBG("process %d is slave\n", getpid()); ret = _ipc_init_slave(name, pktio_entry, pool); } else { - pkt_priv(pktio_entry)->type = PKTIO_TYPE_IPC_MASTER; + pktio_ipc->type = PKTIO_TYPE_IPC_MASTER; snprintf(name, sizeof(name), "%s_info", dev); shm = odp_shm_reserve(name, sizeof(struct pktio_info), ODP_CACHE_LINE_SIZE, ODP_SHM_EXPORT | ODP_SHM_SINGLE_VA); if (ODP_SHM_INVALID == shm) { - _ring_destroy("ipc_rx_cache"); ODP_ERR("can not create shm %s\n", name); return -1; } @@ -541,19 +576,26 @@ static int ipc_pktio_open(odp_pktio_t id ODP_UNUSED, pinfo = odp_shm_addr(shm); pinfo->master.init_done = 0; pinfo->master.pool_name[0] = 0; - pkt_priv(pktio_entry)->pinfo = pinfo; - pkt_priv(pktio_entry)->pinfo_shm = shm; + + pktio_ipc->pinfo = pinfo; + pktio_ipc->pinfo_shm = shm; ODP_DBG("process %d is master\n", getpid()); ret = _ipc_init_master(pktio_entry, dev, pool); } + if (ret) + odp_shm_free(shm); + return ret; } static void _ipc_free_ring_packets(pktio_entry_t *pktio_entry, ring_ptr_t *r, uint32_t r_mask) { - uintptr_t offsets[PKTIO_IPC_ENTRIES]; + pkt_ipc_t *pktio_ipc = pkt_priv(pktio_entry); + uint32_t ring_size = pktio_ipc->ring_size; + uint32_t ring_mask = pktio_ipc->ring_mask; + uintptr_t offsets[ring_size]; int ret; void **rbuf_p; int i; @@ -563,14 +605,13 @@ static void _ipc_free_ring_packets(pktio_entry_t *pktio_entry, ring_ptr_t *r, if (!r) return; - pool = pool_entry_from_hdl(pkt_priv(pktio_entry)->pool); + pool = pool_entry_from_hdl(pktio_ipc->pool); addr = odp_shm_addr(pool->shm); rbuf_p = (void *)&offsets; while (1) { - ret = ring_ptr_deq_multi(r, r_mask, rbuf_p, - PKTIO_IPC_ENTRIES - 1); + ret = ring_ptr_deq_multi(r, r_mask, rbuf_p, ring_mask); if (ret <= 0) break; for (i = 0; i < ret; i++) { @@ -588,34 +629,36 @@ static void _ipc_free_ring_packets(pktio_entry_t *pktio_entry, ring_ptr_t *r, static int ipc_pktio_recv_lockless(pktio_entry_t *pktio_entry, odp_packet_t pkt_table[], int len) { + pkt_ipc_t *pktio_ipc = pkt_priv(pktio_entry); + uint32_t ring_size = pktio_ipc->ring_size; + uint32_t ring_mask = pktio_ipc->ring_mask; int pkts = 0; int i; ring_ptr_t *r; ring_ptr_t *r_p; - uintptr_t offsets[PKTIO_IPC_ENTRIES]; + uintptr_t offsets[ring_size]; void **ipcbufs_p = (void *)&offsets[0]; uint32_t ready; - ready = odp_atomic_load_u32(&pkt_priv(pktio_entry)->ready); + ready = odp_atomic_load_u32(&pktio_ipc->ready); if (odp_unlikely(!ready)) { IPC_ODP_DBG("start pktio is missing before usage?\n"); return 0; } - _ipc_free_ring_packets(pktio_entry, pkt_priv(pktio_entry)->tx.free, - PKTIO_IPC_ENTRY_MASK); + _ipc_free_ring_packets(pktio_entry, pktio_ipc->tx.free, ring_mask); /* rx from cache */ - r = pkt_priv(pktio_entry)->rx.cache; - pkts = ring_ptr_deq_multi(r, PKTIO_IPC_ENTRY_MASK, ipcbufs_p, len); + r = pktio_ipc->rx.cache; + pkts = ring_ptr_deq_multi(r, ring_mask, ipcbufs_p, len); if (odp_unlikely(pkts < 0)) ODP_ABORT("internal error dequeue\n"); /* rx from other app */ if (pkts == 0) { ipcbufs_p = (void *)&offsets[0]; - r = pkt_priv(pktio_entry)->rx.recv; - pkts = ring_ptr_deq_multi(r, PKTIO_IPC_ENTRY_MASK, ipcbufs_p, + r = pktio_ipc->rx.recv; + pkts = ring_ptr_deq_multi(r, ring_mask, ipcbufs_p, len); if (odp_unlikely(pkts < 0)) ODP_ABORT("internal error dequeue\n"); @@ -633,11 +676,10 @@ static int ipc_pktio_recv_lockless(pktio_entry_t *pktio_entry, uint64_t data_pool_off; void *rmt_data_ptr; - phdr = (void *)((uint8_t *)pkt_priv(pktio_entry)-> - pool_mdata_base + - offsets[i]); + phdr = (void *)((uint8_t *)pktio_ipc->pool_mdata_base + + offsets[i]); - pool = pkt_priv(pktio_entry)->pool; + pool = pktio_ipc->pool; if (odp_unlikely(pool == ODP_POOL_INVALID)) ODP_ABORT("invalid pool"); @@ -646,7 +688,7 @@ static int ipc_pktio_recv_lockless(pktio_entry_t *pktio_entry, pkt = odp_packet_alloc(pool, phdr->frame_len); if (odp_unlikely(pkt == ODP_PACKET_INVALID)) { /* Original pool might be smaller then - * PKTIO_IPC_ENTRIES. If packet can not be + * ring size. If packet can not be * allocated from pool at this time, * simple get in on next recv() call. To keep * packet ordering store such packets in local @@ -661,13 +703,11 @@ static int ipc_pktio_recv_lockless(pktio_entry_t *pktio_entry, pkt_data = odp_packet_data(pkt); if (odp_unlikely(!pkt_data)) ODP_ABORT("unable to map pkt_data ipc_slave %d\n", - (PKTIO_TYPE_IPC_SLAVE == - pkt_priv(pktio_entry)->type)); + (PKTIO_TYPE_IPC_SLAVE == pktio_ipc->type)); /* Copy packet data from shared pool to local pool. */ - rmt_data_ptr = (uint8_t *)pkt_priv(pktio_entry)-> - pool_mdata_base + - data_pool_off; + rmt_data_ptr = (uint8_t *)pktio_ipc->pool_mdata_base + + data_pool_off; memcpy(pkt_data, rmt_data_ptr, phdr->frame_len); /* Copy packets L2, L3 parsed offsets and size */ @@ -686,8 +726,8 @@ static int ipc_pktio_recv_lockless(pktio_entry_t *pktio_entry, /* put back to rx ring dequed but not processed packets*/ if (pkts != i) { ipcbufs_p = (void *)&offsets[i]; - r_p = pkt_priv(pktio_entry)->rx.cache; - ring_ptr_enq_multi(r_p, PKTIO_IPC_ENTRY_MASK, ipcbufs_p, + r_p = pktio_ipc->rx.cache; + ring_ptr_enq_multi(r_p, ring_mask, ipcbufs_p, pkts - i); if (i == 0) @@ -698,10 +738,10 @@ static int ipc_pktio_recv_lockless(pktio_entry_t *pktio_entry, pkts = i; /* Now tell other process that we no longer need that buffers.*/ - r_p = pkt_priv(pktio_entry)->rx.free; + r_p = pktio_ipc->rx.free; ipcbufs_p = (void *)&offsets[0]; - ring_ptr_enq_multi(r_p, PKTIO_IPC_ENTRY_MASK, ipcbufs_p, pkts); + ring_ptr_enq_multi(r_p, ring_mask, ipcbufs_p, pkts); for (i = 0; i < pkts; i++) { IPC_ODP_DBG("%d/%d send to be free packet offset %x\n", @@ -728,11 +768,13 @@ static int ipc_pktio_recv(pktio_entry_t *pktio_entry, int index ODP_UNUSED, static int ipc_pktio_send_lockless(pktio_entry_t *pktio_entry, const odp_packet_t pkt_table[], int num) { + pkt_ipc_t *pktio_ipc = pkt_priv(pktio_entry); + uint32_t ring_mask = pktio_ipc->ring_mask; ring_ptr_t *r; void **rbuf_p; int i; - uint32_t ready = odp_atomic_load_u32(&pkt_priv(pktio_entry)->ready); - pool_t *ipc_pool = pool_entry_from_hdl(pkt_priv(pktio_entry)->pool); + uint32_t ready = odp_atomic_load_u32(&pktio_ipc->ready); + pool_t *ipc_pool = pool_entry_from_hdl(pktio_ipc->pool); odp_packet_t pkt_table_mapped[num]; /**< Ready to send packet has to be * in memory mapped pool. */ uintptr_t offsets[num]; @@ -740,8 +782,7 @@ static int ipc_pktio_send_lockless(pktio_entry_t *pktio_entry, if (odp_unlikely(!ready)) return 0; - _ipc_free_ring_packets(pktio_entry, pkt_priv(pktio_entry)->tx.free, - PKTIO_IPC_ENTRY_MASK); + _ipc_free_ring_packets(pktio_entry, pktio_ipc->tx.free, ring_mask); /* Copy packets to shm shared pool if they are in different * pool, or if they are references (we can't share across IPC). @@ -758,8 +799,7 @@ static int ipc_pktio_send_lockless(pktio_entry_t *pktio_entry, odp_packet_has_ref(pkt)) { odp_packet_t newpkt; - newpkt = odp_packet_copy(pkt, - pkt_priv(pktio_entry)->pool); + newpkt = odp_packet_copy(pkt, pktio_ipc->pool); if (newpkt == ODP_PACKET_INVALID) ODP_ABORT("Unable to copy packet\n"); @@ -791,14 +831,13 @@ static int ipc_pktio_send_lockless(pktio_entry_t *pktio_entry, odp_packet_to_u64(pkt), odp_pool_to_u64(pool_hdl), pkt_hdr, pkt_hdr->buf_hdr.ipc_data_offset, offsets[i], odp_shm_addr(pool->shm), - odp_shm_addr(pool_entry_from_hdl( - pkt_priv(pktio_entry)->pool)->shm)); + odp_shm_addr(ipc_pool->shm)); } /* Put packets to ring to be processed by other process. */ rbuf_p = (void *)&offsets[0]; - r = pkt_priv(pktio_entry)->tx.send; - ring_ptr_enq_multi(r, PKTIO_IPC_ENTRY_MASK, rbuf_p, num); + r = pktio_ipc->tx.send; + ring_ptr_enq_multi(r, ring_mask, rbuf_p, num); return num; } @@ -847,19 +886,20 @@ static int ipc_start(pktio_entry_t *pktio_entry) static int ipc_stop(pktio_entry_t *pktio_entry) { - odp_atomic_store_u32(&pkt_priv(pktio_entry)->ready, 0); + pkt_ipc_t *pktio_ipc = pkt_priv(pktio_entry); + uint32_t ring_mask = pktio_ipc->ring_mask; + + odp_atomic_store_u32(&pktio_ipc->ready, 0); - if (pkt_priv(pktio_entry)->tx.send) - _ipc_free_ring_packets(pktio_entry, - pkt_priv(pktio_entry)->tx.send, - PKTIO_IPC_ENTRY_MASK); + if (pktio_ipc->tx.send) + _ipc_free_ring_packets(pktio_entry, pktio_ipc->tx.send, + ring_mask); /* other process can transfer packets from one ring to * other, use delay here to free that packets. */ sleep(1); - if (pkt_priv(pktio_entry)->tx.free) - _ipc_free_ring_packets(pktio_entry, - pkt_priv(pktio_entry)->tx.free, - PKTIO_IPC_ENTRY_MASK); + if (pktio_ipc->tx.free) + _ipc_free_ring_packets(pktio_entry, pktio_ipc->tx.free, + ring_mask); return 0; } commit b89a5b77e92060c9343ab162526d05a8ee97f8b4 Author: Matias Elo <matias.elo(a)nokia.com> Date: Fri Oct 25 13:59:58 2019 +0300 linux-gen: ipc: clean up code Reorder code and remove unused variables and defines. Signed-off-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen(a)nokia.com> diff --git a/platform/linux-generic/pktio/ipc.c b/platform/linux-generic/pktio/ipc.c index 00edcdb7e..dec918e95 100644 --- a/platform/linux-generic/pktio/ipc.c +++ b/platform/linux-generic/pktio/ipc.c @@ -11,6 +11,7 @@ #include <odp/api/system_info.h> #include <odp_shm_internal.h> #include <odp_ring_ptr_internal.h> +#include <odp_global_data.h> #include <errno.h> #include <fcntl.h> @@ -38,15 +39,12 @@ struct pktio_info { struct { /* number of buffer*/ int num; - /* size of packet/segment in remote pool */ - uint32_t block_size; char pool_name[ODP_POOL_NAME_LEN]; /* 1 if master finished creation of all shared objects */ int init_done; } master; struct { void *base_addr; - uint32_t block_size; char pool_name[ODP_POOL_NAME_LEN]; /* pid of the slave process written to shm and * used by master to look up memory created by @@ -57,46 +55,79 @@ struct pktio_info { } slave; } ODP_PACKED; -/* The maximum length of a ring name. */ -#define _RING_NAMESIZE 32 -/* If set - ring is visible from different processes. - * Default is thread visible.*/ -#define _RING_SHM_PROC (1 << 2) -/* Ring size mask */ -#define _RING_SZ_MASK (unsigned)(0x0fffffff) +typedef struct { + /* TX */ + struct { + /* ODP ring for IPC msg packets indexes transmitted to shared + * memory */ + ring_ptr_t *send; + /* ODP ring for IPC msg packets indexes already processed by + * remote process */ + ring_ptr_t *free; + } tx; + /* RX */ + struct { + /* ODP ring for IPC msg packets indexes received from shared + * memory (from remote process) */ + ring_ptr_t *recv; + /* odp ring for ipc msg packets indexes already processed by + * current process */ + ring_ptr_t *free; + /* local cache to keep packet order right */ + ring_ptr_t *cache; + } rx; /* slave */ + void *pool_base; /**< Remote pool base addr */ + void *pool_mdata_base; /**< Remote pool mdata base addr */ + odp_pool_t pool; /**< Pool of main process */ + enum { + PKTIO_TYPE_IPC_MASTER = 0, /**< Master is the process which + creates shm */ + PKTIO_TYPE_IPC_SLAVE /**< Slave is the process which + connects to shm */ + } type; /**< define if it's master or slave process */ + odp_atomic_u32_t ready; /**< 1 - pktio is ready and can recv/send + packet, 0 - not yet ready */ + void *pinfo; + odp_shm_t pinfo_shm; + odp_shm_t remote_pool_shm; /**< shm of remote pool get with + _ipc_map_remote_pool() */ +} pkt_ipc_t; + +ODP_STATIC_ASSERT(PKTIO_PRIVATE_SIZE >= sizeof(pkt_ipc_t), + "PKTIO_PRIVATE_SIZE too small"); + +static inline pkt_ipc_t *pkt_priv(pktio_entry_t *pktio_entry) +{ + return (pkt_ipc_t *)(uintptr_t)(pktio_entry->s.pkt_priv); +} + +/* MAC address for the "ipc" interface */ +static const char pktio_ipc_mac[] = {0x12, 0x12, 0x12, 0x12, 0x12, 0x12}; + +static odp_shm_t _ipc_map_remote_pool(const char *name, int pid); /* create the ring */ -static ring_ptr_t * -_ring_create(const char *name, unsigned count, unsigned flags) +static ring_ptr_t *_ring_create(const char *name, uint32_t count, + uint32_t shm_flags) { - char ring_name[_RING_NAMESIZE]; ring_ptr_t *r; size_t ring_size; - uint32_t shm_flag; odp_shm_t shm; - if (flags & _RING_SHM_PROC) - shm_flag = ODP_SHM_PROC | ODP_SHM_EXPORT; - else - shm_flag = 0; if (odp_global_ro.shm_single_va) - shm_flag |= ODP_SHM_SINGLE_VA; + shm_flags |= ODP_SHM_SINGLE_VA; /* count must be a power of 2 */ if (!CHECK_IS_POWER2(count)) { - ODP_ERR("Requested size is invalid, must be power of 2," - "and do not exceed the size limit %u\n", - _RING_SZ_MASK); + ODP_ERR("Requested size is invalid, must be a power of 2\n"); __odp_errno = EINVAL; return NULL; } - snprintf(ring_name, sizeof(ring_name), "%s", name); ring_size = sizeof(ring_ptr_t) + count * sizeof(void *); /* reserve a memory zone for this ring.*/ - shm = odp_shm_reserve(ring_name, ring_size, ODP_CACHE_LINE_SIZE, - shm_flag); + shm = odp_shm_reserve(name, ring_size, ODP_CACHE_LINE_SIZE, shm_flags); r = odp_shm_addr(shm); if (r != NULL) { @@ -124,7 +155,7 @@ static int _ring_destroy(const char *name) /** * Return the number of entries in a ring. */ -static unsigned _ring_count(ring_ptr_t *r, uint32_t mask) +static uint32_t _ring_count(ring_ptr_t *r, uint32_t mask) { uint32_t prod_tail = odp_atomic_load_u32(&r->r.w_tail); uint32_t cons_tail = odp_atomic_load_u32(&r->r.r_tail); @@ -135,7 +166,7 @@ static unsigned _ring_count(ring_ptr_t *r, uint32_t mask) /** * Return the number of free entries in a ring. */ -static unsigned _ring_free_count(ring_ptr_t *r, uint32_t mask) +static uint32_t _ring_free_count(ring_ptr_t *r, uint32_t mask) { uint32_t prod_tail = odp_atomic_load_u32(&r->r.w_tail); uint32_t cons_tail = odp_atomic_load_u32(&r->r.r_tail); @@ -143,58 +174,6 @@ static unsigned _ring_free_count(ring_ptr_t *r, uint32_t mask) return (cons_tail - prod_tail - 1) & mask; } -typedef struct { - /* TX */ - struct { - /* ODP ring for IPC msg packets indexes transmitted to shared - * memory */ - ring_ptr_t *send; - /* ODP ring for IPC msg packets indexes already processed by - * remote process */ - ring_ptr_t *free; - } tx; - /* RX */ - struct { - /* ODP ring for IPC msg packets indexes received from shared - * memory (from remote process) */ - ring_ptr_t *recv; - /* odp ring for ipc msg packets indexes already processed by - * current process */ - ring_ptr_t *free; - /* local cache to keep packet order right */ - ring_ptr_t *cache; - } rx; /* slave */ - void *pool_base; /**< Remote pool base addr */ - void *pool_mdata_base; /**< Remote pool mdata base addr */ - uint64_t pkt_size; /**< Packet size in remote pool */ - odp_pool_t pool; /**< Pool of main process */ - enum { - PKTIO_TYPE_IPC_MASTER = 0, /**< Master is the process which - creates shm */ - PKTIO_TYPE_IPC_SLAVE /**< Slave is the process which - connects to shm */ - } type; /**< define if it's master or slave process */ - odp_atomic_u32_t ready; /**< 1 - pktio is ready and can recv/send - packet, 0 - not yet ready */ - void *pinfo; - odp_shm_t pinfo_shm; - odp_shm_t remote_pool_shm; /**< shm of remote pool get with - _ipc_map_remote_pool() */ -} pkt_ipc_t; - -ODP_STATIC_ASSERT(PKTIO_PRIVATE_SIZE >= sizeof(pkt_ipc_t), - "PKTIO_PRIVATE_SIZE too small"); - -static inline pkt_ipc_t *pkt_priv(pktio_entry_t *pktio_entry) -{ - return (pkt_ipc_t *)(uintptr_t)(pktio_entry->s.pkt_priv); -} - -/* MAC address for the "ipc" interface */ -static const char pktio_ipc_mac[] = {0x12, 0x12, 0x12, 0x12, 0x12, 0x12}; - -static odp_shm_t _ipc_map_remote_pool(const char *name, int pid); - static const char *_ipc_odp_buffer_pool_shm_name(odp_pool_t pool_hdl) { pool_t *pool; @@ -240,13 +219,9 @@ static int _ipc_init_master(pktio_entry_t *pktio_entry, odp_pool_t pool_hdl) { char ipc_shm_name[ODP_POOL_NAME_LEN + sizeof("_m_prod")]; - pool_t *pool; struct pktio_info *pinfo; const char *pool_name; - pool = pool_entry_from_hdl(pool_hdl); - (void)pool; - if (strlen(dev) > (ODP_POOL_NAME_LEN - sizeof("_m_prod"))) { ODP_ERR("too big ipc name\n"); return -1; @@ -258,7 +233,8 @@ static int _ipc_init_master(pktio_entry_t *pktio_entry, snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_m_prod", dev); pkt_priv(pktio_entry)->tx.send = _ring_create(ipc_shm_name, PKTIO_IPC_ENTRIES, - _RING_SHM_PROC); + ODP_SHM_PROC | + ODP_SHM_EXPORT); if (!pkt_priv(pktio_entry)->tx.send) { ODP_ERR("pid %d unable to create ipc ring %s name\n", getpid(), ipc_shm_name); @@ -276,7 +252,8 @@ static int _ipc_init_master(pktio_entry_t *pktio_entry, snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_m_cons", dev); pkt_priv(pktio_entry)->tx.free = _ring_create(ipc_shm_name, PKTIO_IPC_ENTRIES, - _RING_SHM_PROC); + ODP_SHM_PROC | + ODP_SHM_EXPORT); if (!pkt_priv(pktio_entry)->tx.free) { ODP_ERR("pid %d unable to create ipc ring %s name\n", getpid(), ipc_shm_name); @@ -291,7 +268,8 @@ static int _ipc_init_master(pktio_entry_t *pktio_entry, snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_s_prod", dev); pkt_priv(pktio_entry)->rx.recv = _ring_create(ipc_shm_name, PKTIO_IPC_ENTRIES, - _RING_SHM_PROC); + ODP_SHM_PROC | + ODP_SHM_EXPORT); if (!pkt_priv(pktio_entry)->rx.recv) { ODP_ERR("pid %d unable to create ipc ring %s name\n", getpid(), ipc_shm_name); @@ -306,7 +284,8 @@ static int _ipc_init_master(pktio_entry_t *pktio_entry, snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_s_cons", dev); pkt_priv(pktio_entry)->rx.free = _ring_create(ipc_shm_name, PKTIO_IPC_ENTRIES, - _RING_SHM_PROC); + ODP_SHM_PROC | + ODP_SHM_EXPORT); if (!pkt_priv(pktio_entry)->rx.free) { ODP_ERR("pid %d unable to create ipc ring %s name\n", getpid(), ipc_shm_name); @@ -361,7 +340,6 @@ static void _ipc_export_pool(struct pktio_info *pinfo, snprintf(pinfo->slave.pool_name, ODP_POOL_NAME_LEN, "%s", _ipc_odp_buffer_pool_shm_name(pool_hdl)); pinfo->slave.pid = odp_global_ro.main_pid; - pinfo->slave.block_size = pool->block_size; pinfo->slave.base_addr = pool->base_addr; } @@ -480,7 +458,6 @@ static int _ipc_slave_start(pktio_entry_t *pktio_entry) pid); pkt_priv(pktio_entry)->remote_pool_shm = shm; pkt_priv(pktio_entry)->pool_mdata_base = (char *)odp_shm_addr(shm); - pkt_priv(pktio_entry)->pkt_size = pinfo->master.block_size; _ipc_export_pool(pinfo, pkt_priv(pktio_entry)->pool); @@ -517,9 +494,6 @@ static int ipc_pktio_open(odp_pktio_t id ODP_UNUSED, char tail[ODP_POOL_NAME_LEN]; odp_shm_t shm; - ODP_STATIC_ASSERT(ODP_POOL_NAME_LEN == _RING_NAMESIZE, - "mismatch pool and ring name arrays"); - if (strncmp(dev, "ipc", 3)) return -1; commit fc07575511c3a5b79ccae80ca278b3daad4f101d Author: Matias Elo <matias.elo(a)nokia.com> Date: Fri Oct 25 13:01:00 2019 +0300 linux-gen: ipc: remove now unnecessary init/term functions Signed-off-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen(a)nokia.com> diff --git a/platform/linux-generic/pktio/ipc.c b/platform/linux-generic/pktio/ipc.c index faabe8a51..00edcdb7e 100644 --- a/platform/linux-generic/pktio/ipc.c +++ b/platform/linux-generic/pktio/ipc.c @@ -65,43 +65,6 @@ struct pktio_info { /* Ring size mask */ #define _RING_SZ_MASK (unsigned)(0x0fffffff) -typedef struct { - /* Rings tailq lock */ - odp_rwlock_t qlock; - odp_shm_t shm; -} global_data_t; - -static global_data_t *global; - -/* Initialize tailq_ring */ -static int _ring_global_init(void) -{ - odp_shm_t shm; - - /* Allocate globally shared memory */ - shm = odp_shm_reserve("_odp_ring_global", sizeof(global_data_t), - ODP_CACHE_LINE_SIZE, 0); - if (ODP_SHM_INVALID == shm) { - ODP_ERR("Shm reserve failed for pktio ring\n"); - return -1; - } - - global = odp_shm_addr(shm); - memset(global, 0, sizeof(global_data_t)); - global->shm = shm; - - return 0; -} - -static int _ring_global_term(void) -{ - if (odp_shm_free(global->shm)) { - ODP_ERR("Shm free failed for pktio ring\n"); - return -1; - } - return 0; -} - /* create the ring */ static ring_ptr_t * _ring_create(const char *name, unsigned count, unsigned flags) @@ -961,23 +924,12 @@ static int ipc_close(pktio_entry_t *pktio_entry) return 0; } -static int ipc_pktio_init_global(void) -{ - ODP_DBG("PKTIO: initializing ipc interface.\n"); - return _ring_global_init(); -} - -static int ipc_pktio_term_global(void) -{ - return _ring_global_term(); -} - const pktio_if_ops_t ipc_pktio_ops = { .name = "ipc", .print = NULL, - .init_global = ipc_pktio_init_global, + .init_global = NULL, .init_local = NULL, - .term = ipc_pktio_term_global, + .term = NULL, .open = ipc_pktio_open, .close = ipc_close, .recv = ipc_pktio_recv, commit ff99ba52f99a608dc1a41f4d8a86ab5c2f8491ca Author: Matias Elo <matias.elo(a)nokia.com> Date: Fri Oct 25 11:13:59 2019 +0300 linux-gen: ipc: remove unnecessary odp_packet_io_ipc_internal.h header Copy the remaining necessary code from odp_packet_io_ipc_internal.h to ipc.c. Signed-off-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen(a)nokia.com> diff --git a/platform/linux-generic/Makefile.am b/platform/linux-generic/Makefile.am index a3ec92af6..0d15e53d9 100644 --- a/platform/linux-generic/Makefile.am +++ b/platform/linux-generic/Makefile.am @@ -108,7 +108,6 @@ noinst_HEADERS = \ include/odp_packet_dpdk.h \ include/odp_packet_internal.h \ include/odp_packet_io_internal.h \ - include/odp_packet_io_ipc_internal.h \ include/odp_socket_common.h \ include/odp_packet_io_stats_common.h \ include/odp_packet_io_stats.h \ diff --git a/platform/linux-generic/include/odp_packet_io_ipc_internal.h b/platform/linux-generic/include/odp_packet_io_ipc_internal.h deleted file mode 100644 index 8962f8936..000000000 --- a/platform/linux-generic/include/odp_packet_io_ipc_internal.h +++ /dev/null @@ -1,46 +0,0 @@ -/* Copyright (c) 2015-2018, Linaro Limited - * All rights reserved. - * - * SPDX-License-Identifier: BSD-3-Clause - */ - -#include <odp/api/packet_io.h> -#include <odp_packet_io_internal.h> -#include <odp/api/packet.h> -#include <odp_packet_internal.h> -#include <odp/api/shared_memory.h> - -#include <string.h> -#include <unistd.h> -#include <stdlib.h> - -/* number of odp buffers in odp ring queue */ -#define PKTIO_IPC_ENTRIES 4096 - -#define PKTIO_IPC_ENTRY_MASK (PKTIO_IPC_ENTRIES - 1) - -/* that struct is exported to shared memory, so that processes can find - * each other. - */ -struct pktio_info { - struct { - /* number of buffer*/ - int num; - /* size of packet/segment in remote pool */ - uint32_t block_size; - char pool_name[ODP_POOL_NAME_LEN]; - /* 1 if master finished creation of all shared objects */ - int init_done; - } master; - struct { - void *base_addr; - uint32_t block_size; - char pool_name[ODP_POOL_NAME_LEN]; - /* pid of the slave process written to shm and - * used by master to look up memory created by - * slave - */ - int pid; - int init_done; - } slave; -} ODP_PACKED; diff --git a/platform/linux-generic/odp_packet_io.c b/platform/linux-generic/odp_packet_io.c index 6e4f36569..33ba8fdb4 100644 --- a/platform/linux-generic/odp_packet_io.c +++ b/platform/linux-generic/odp_packet_io.c @@ -24,7 +24,6 @@ #include <odp_schedule_if.h> #include <odp_classification_internal.h> #include <odp_debug_internal.h> -#include <odp_packet_io_ipc_internal.h> #include <odp/api/time.h> #include <odp/api/plat/time_inlines.h> #include <odp_pcapng.h> diff --git a/platform/linux-generic/pktio/ipc.c b/platform/linux-generic/pktio/ipc.c index 680bfa392..faabe8a51 100644 --- a/platform/linux-generic/pktio/ipc.c +++ b/platform/linux-generic/pktio/ipc.c @@ -5,7 +5,6 @@ * SPDX-License-Identifier: BSD-3-Clause */ -#include <odp_packet_io_ipc_internal.h> #include <odp_debug_internal.h> #include <odp_packet_io_internal.h> #include <odp_errno_define.h> @@ -17,6 +16,7 @@ #include <fcntl.h> #include <sys/mman.h> #include <sys/stat.h> +#include <unistd.h> #define IPC_ODP_DEBUG_PRINT 0 @@ -26,6 +26,37 @@ ODP_DBG(fmt, ##__VA_ARGS__);\ } while (0) +/* number of odp buffers in odp ring queue */ +#define PKTIO_IPC_ENTRIES 4096 + +#define PKTIO_IPC_ENTRY_MASK (PKTIO_IPC_ENTRIES - 1) + +/* that struct is exported to shared memory, so that processes can find + * each other. + */ +struct pktio_info { + struct { + /* number of buffer*/ + int num; + /* size of packet/segment in remote pool */ + uint32_t block_size; + char pool_name[ODP_POOL_NAME_LEN]; + /* 1 if master finished creation of all shared objects */ + int init_done; + } master; + struct { + void *base_addr; + uint32_t block_size; + char pool_name[ODP_POOL_NAME_LEN]; + /* pid of the slave process written to shm and + * used by master to look up memory created by + * slave + */ + int pid; + int init_done; + } slave; +} ODP_PACKED; + /* The maximum length of a ring name. */ #define _RING_NAMESIZE 32 /* If set - ring is visible from different processes. commit 426ee8c61835c8faa5df0873ecc4ec7c8667c691 Author: Matias Elo <matias.elo(a)nokia.com> Date: Fri Oct 25 11:08:52 2019 +0300 linux-gen: ipc: move remaining ring helper functions to ipc source file The remaining ring helper functions are only used by the IPC pktio, so move them to ipc.c. Signed-off-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen(a)nokia.com> diff --git a/platform/linux-generic/Makefile.am b/platform/linux-generic/Makefile.am index 0d4d905f9..a3ec92af6 100644 --- a/platform/linux-generic/Makefile.am +++ b/platform/linux-generic/Makefile.am @@ -109,7 +109,6 @@ noinst_HEADERS = \ include/odp_packet_internal.h \ include/odp_packet_io_internal.h \ include/odp_packet_io_ipc_internal.h \ - include/odp_packet_io_ring_internal.h \ include/odp_socket_common.h \ include/odp_packet_io_stats_common.h \ include/odp_packet_io_stats.h \ @@ -224,7 +223,6 @@ __LIB__libodp_linux_la_SOURCES = \ pktio/netmap.c \ pktio/null.c \ pktio/pktio_common.c \ - pktio/ring.c \ pktio/socket.c \ pktio/socket_mmap.c \ pktio/tap.c diff --git a/platform/linux-generic/include/odp_packet_io_internal.h b/platform/linux-generic/include/odp_packet_io_internal.h index d12db8339..ba2e7c6bc 100644 --- a/platform/linux-generic/include/odp_packet_io_internal.h +++ b/platform/linux-generic/include/odp_packet_io_internal.h @@ -27,7 +27,6 @@ extern "C" { #include <odp_classification_datamodel.h> #include <odp_align_internal.h> #include <odp_debug_internal.h> -#include <odp_packet_io_ring_internal.h> #include <odp_packet_io_stats_common.h> #include <odp_queue_if.h> diff --git a/platform/linux-generic/include/odp_packet_io_ipc_internal.h b/platform/linux-generic/include/odp_packet_io_ipc_internal.h index 59c268cb9..8962f8936 100644 --- a/platform/linux-generic/include/odp_packet_io_ipc_internal.h +++ b/platform/linux-generic/include/odp_packet_io_ipc_internal.h @@ -14,9 +14,6 @@ #include <unistd.h> #include <stdlib.h> -/* IPC packet I/O over shared memory ring */ -#include <odp_packet_io_ring_internal.h> - /* number of odp buffers in odp ring queue */ #define PKTIO_IPC_ENTRIES 4096 diff --git a/platform/linux-generic/include/odp_packet_io_ring_internal.h b/platform/linux-generic/include/odp_packet_io_ring_internal.h deleted file mode 100644 index 22f1ae017..000000000 --- a/platform/linux-generic/include/odp_packet_io_ring_internal.h +++ /dev/null @@ -1,413 +0,0 @@ -/* Copyright (c) 2014-2018, Linaro Limited - * All rights reserved. - * - * SPDX-License-Identifier: BSD-3-Clause - */ - -/*- - * BSD LICENSE - * - * Copyright(c) 2010-2013 Intel Corporation. All rights reserved. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in - * the documentation and/or other materials provided with the - * distribution. - * * Neither the name of Intel Corporation nor the names of its - * contributors may be used to endorse or promote products derived - * from this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - */ - -/* - * Derived from FreeBSD's bufring.c - * - ************************************************************************** - * - * Copyright (c) 2007,2008 Kip Macy kmacy(a)freebsd.org - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * 1. Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * - * 2. The name of Kip Macy nor the names of other - * contributors may be used to endorse or promote products derived from - * this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" - * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE - * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR - * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS - * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN - * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE - * POSSIBILITY OF SUCH DAMAGE. - * - ***************************************************************************/ - -/** - * ODP Ring - * - * The Ring Manager is a fixed-size queue, implemented as a table of - * pointers. Head and tail pointers are modified atomically, allowing - * concurrent access to it. It has the following features: - * - * - FIFO (First In First Out) - * - Maximum size is fixed; the pointers are stored in a table. - * - Lockless implementation. - * - Multi- or single-consumer dequeue. - * - Multi- or single-producer enqueue. - * - Bulk dequeue. - * - Bulk enqueue. - * - * Note: the ring implementation is not preemptable. A lcore must not - * be interrupted by another task that uses the same ring. - * - */ - -#ifndef _RING_H_ -#define _RING_H_ - -#ifdef __cplusplus -extern "C" { -#endif - -#include <odp/api/std_types.h> -#include <odp/api/hints.h> -#include <odp/api/atomic.h> -#include <errno.h> -#include <sys/queue.h> -#include <odp_debug_internal.h> - -#include <odp_ring_ptr_internal.h> - -enum _ring_queue_behavior { - _RING_QUEUE_FIXED = 0, /**< Enq/Deq a fixed number - of items from a ring */ - _RING_QUEUE_VARIABLE /**< Enq/Deq as many items - a possible from ring */ -}; - -#define _RING_NAMESIZE 32 /**< The maximum length of a ring name. */ - -/** - * An ODP ring structure. - * - * The producer and the consumer have a head and a tail index. The particularity - * of these index is that they are not between 0 and size(ring). These indexes - * are between 0 and 2^32, and we mask their value when we access the ring[] - * field. Thanks to this assumption, we can do subtractions between 2 index - * values in a modulo-32bit base: that's why the overflow of the indexes is not - * a problem. - */ -typedef struct _ring { - /** @private Next in list. */ - TAILQ_ENTRY(_ring) next; - - /** @private Name of the ring. */ - char name[_RING_NAMESIZE]; - /** @private Flags supplied at creation. */ - int flags; - - /** @private Producer */ - struct ODP_ALIGNED_CACHE _prod { - uint32_t size; /* Size of ring. */ - uint32_t mask; /* Mask (size-1) of ring. */ - volatile uint32_t head; /* Producer head. */ - volatile uint32_t tail; /* Producer tail. */ - } prod; - - /** @private Consumer */ - struct ODP_ALIGNED_CACHE _cons { - uint32_t size; /* Size of the ring. */ - uint32_t mask; /* Mask (size-1) of ring. */ - volatile uint32_t head; /* Consumer head. */ - volatile uint32_t tail; /* Consumer tail. */ - } cons; - - /** @private Memory space of ring starts here. */ - void ODP_ALIGNED_CACHE *ring[0]; -} _ring_t; - -/* The default enqueue is "single-producer".*/ -#define _RING_F_SP_ENQ (1 << 0) -/* The default dequeue is "single-consumer".*/ -#define _RING_F_SC_DEQ (1 << 1) -/* If set - ring is visible from different processes. - * Default is thread visible.*/ -#define _RING_SHM_PROC (1 << 2) -/* Ring size mask */ -#define _RING_SZ_MASK (unsigned)(0x0fffffff) - -/** - * Create a new ring named *name* in memory. - * - * This function uses odp_shm_reserve() to allocate memory. Its size is - * set to *count*, which must be a power of two. Note that the real usable - * ring size is count-1 instead of count. - * - * @param name - * The name of the ring. - * @param count - * The size of the ring (must be a power of 2). - * @param flags - * An OR of the following: - * - RING_F_SP_ENQ: If this flag is set, the default behavior when - * using ``odph_ring_enqueue()`` or ``odph_ring_enqueue_bulk()`` - * is "single-producer". Otherwise, it is "multi-producers". - * - RING_F_SC_DEQ: If this flag is set, the default behavior when - * using ``odph_ring_dequeue()`` or ``odph_ring_dequeue_bulk()`` - * is "single-consumer". Otherwise, it is "multi-consumers". - * @return - * On success, the pointer to the new allocated ring. NULL on error with - * odp_errno set appropriately. Possible errno values include: - * - EINVAL - count provided is not a power of 2 - * - ENOSPC - the maximum number of memzones has already been allocated - * - EEXIST - a memzone with the same name already exists - * - ENOMEM - no appropriate memory area found in which to create memzone - */ -ring_ptr_t *_ring_create(const char *name, unsigned count, unsigned flags); - -/** - * Destroy the ring created with *name*. - * - * @param name name of the ring to be destroyed. - * @return 0 on success and negative value on error. - */ -int _ring_destroy(const char *name); - -/** - * Dump the status of the ring to the console. - * - * @param r A pointer to the ring structure. - */ -void _ring_dump(ring_ptr_t *r, uint32_t mask); - -/** - * Enqueue several objects on the ring (multi-producers safe). - * - * This function uses a "compare and set" instruction to move the - * producer index atomically. - * - * @param r - * A pointer to the ring structure. - * @param obj_table - * A pointer to a table of void * pointers (objects). - * @param n - * The number of objects to add in the ring from the obj_table. - * @param behavior - * ODPH_RING_QUEUE_FIXED: Enqueue a fixed number of items from a ring - * ODPH_RING_QUEUE_VARIABLE: Enqueue as many items a possible from ring - * @return - * Depend on the behavior value - * if behavior = ODPH_RING_QUEUE_FIXED - * - 0: Success; objects enqueue. - * - -ENOBUFS: Not enough room in the ring to enqueue, no object is enqueued. - * if behavior = ODPH_RING_QUEUE_VARIABLE - * - n: Actual number of objects enqueued. - */ -int ___ring_mp_do_enqueue(ring_ptr_t *r, void * const *obj_table, - unsigned n, - enum _ring_queue_behavior behavior); - -/** - * Dequeue several objects from a ring (multi-consumers safe). When - * the request objects are more than the available objects, only dequeue the - * actual number of objects - * - * This function uses a "compare and set" instruction to move the - * consumer index atomically. - * - * @param r - * A pointer to the ring structure. - * @param obj_table - * A pointer to a table of void * pointers (objects) that will be filled. - * @param n - * The number of objects to dequeue from the ring to the obj_table. - * @param behavior - * ODPH_RING_QUEUE_FIXED: Dequeue a fixed number of items from a ring - * ODPH_RING_QUEUE_VARIABLE: Dequeue as many items a possible from ring - * @return - * Depend on the behavior value - * if behavior = ODPH_RING_QUEUE_FIXED - * - 0: Success; objects dequeued. - * - -ENOENT: Not enough entries in the ring to dequeue; no object is - * dequeued. - * if behavior = ODPH_RING_QUEUE_VARIABLE - * - n: Actual number of objects dequeued. - */ - -int ___ring_mc_do_dequeue(ring_ptr_t *r, void **obj_table, - unsigned n, - enum _ring_queue_behavior behavior); - -/** - * Enqueue several objects on the ring (multi-producers safe). - * - * This function uses a "compare and set" instruction to move the - * producer index atomically. - * - * @param r - * A pointer to the ring structure. - * @param obj_table - * A pointer to a table of void * pointers (objects). - * @param n - * The number of objects to add in the ring from the obj_table. - * @return - * - 0: Success; objects enqueue. - * - -EDQUOT: Quota exceeded. The objects have been enqueued, but the - * high water mark is exceeded. - * - -ENOBUFS: Not enough room in the ring to enqueue, no object is enqueued. - */ -int _ring_mp_enqueue_bulk(ring_ptr_t *r, void * const *obj_table, - unsigned n); - -/** - * Dequeue several objects from a ring (multi-consumers safe). - * - * This function uses a "compare and set" instruction to move the - * consumer index atomically. - * - * @param r - * A pointer to the ring structure. - * @param obj_table - * A pointer to a table of void * pointers (objects) that will be filled. - * @param n - * The number of objects to dequeue from the ring to the obj_table. - * @return - * - 0: Success; objects dequeued. - * - -ENOENT: Not enough entries in the ring to dequeue; no object is - * dequeued. - */ -int _ring_mc_dequeue_bulk(ring_ptr_t *r, void **obj_table, unsigned n); - -/** - * Test if a ring is full. - * - * @param r - * A pointer to the ring structure. - * @return - * - 1: The ring is full. - * - 0: The ring is not full. - */ -int _ring_full(ring_ptr_t *r, uint32_t mask); - -/** - * Test if a ring is empty. - * - * @param r - * A pointer to the ring structure. - * @return - * - 1: The ring is empty. - * - 0: The ring is not empty. - */ -int _ring_empty(ring_ptr_t *r, uint32_t mask); - -/** - * Return the number of entries in a ring. - * - * @param r - * A pointer to the ring structure. - * @return - * The number of entries in the ring. - */ -unsigned _ring_count(ring_ptr_t *r, uint32_t mask); - -/** - * Return the number of free entries in a ring. - * - * @param r - * A pointer to the ring structure. - * @return - * The number of free entries in the ring. - */ -unsigned _ring_free_count(ring_ptr_t *r, uint32_t mask); - -/** - * search ring by name - * @param name ring name to search - * @return pointer to ring otherwise NULL - */ -ring_ptr_t *_ring_lookup(const char *name); - -/** - * Enqueue several objects on the ring (multi-producers safe). - * - * This function uses a "compare and set" instruction to move the - * producer index atomically. - * - * @param r - * A pointer to the ring structure. - * @param obj_table - * A pointer to a table of void * pointers (objects). - * @param n - * The number of objects to add in the ring from the obj_table. - * @return - * - n: Actual number of objects enqueued. - */ -int _ring_mp_enqueue_burst(ring_ptr_t *r, void * const *obj_table, - unsigned n); - -/** - * Dequeue several objects from a ring (multi-consumers safe). When the request - * objects are more than the available objects, only dequeue the actual number - * of objects - * - * This function uses a "compare and set" instruction to move the - * consumer index atomically. - * - * @param r - * A pointer to the ring structure. - * @param obj_table - * A pointer to a table of void * pointers (objects) that will be filled. - * @param n - * The number of objects to dequeue from the ring to the obj_table. - * @return - * - n: Actual number of objects dequeued, 0 if ring is empty - */ -int _ring_mc_dequeue_burst(ring_ptr_t *r, void **obj_table, unsigned n); - -/** - * dump the status of all rings on the console - */ -void _ring_list_dump(void); - -/** - * Initialize ring tailq - */ -int _ring_global_init(void); - -/** - * Terminate ring tailq - */ -int _ring_global_term(void); - -#ifdef __cplusplus -} -#endif - -#endif diff --git a/platform/linux-generic/pktio/ipc.c b/platform/linux-generic/pktio/ipc.c index a7678f261..680bfa392 100644 --- a/platform/linux-generic/pktio/ipc.c +++ b/platform/linux-generic/pktio/ipc.c @@ -1,4 +1,5 @@ /* Copyright (c) 2015-2018, Linaro Limited + * Copyright (c) 2019, Nokia * All rights reserved. * * SPDX-License-Identifier: BSD-3-Clause @@ -7,13 +8,15 @@ #include <odp_packet_io_ipc_internal.h> #include <odp_debug_internal.h> #include <odp_packet_io_internal.h> +#include <odp_errno_define.h> #include <odp/api/system_info.h> #include <odp_shm_internal.h> #include <odp_ring_ptr_internal.h> +#include <errno.h> +#include <fcntl.h> #include <sys/mman.h> #include <sys/stat.h> -#include <fcntl.h> #define IPC_ODP_DEBUG_PRINT 0 @@ -23,6 +26,129 @@ ODP_DBG(fmt, ##__VA_ARGS__);\ } while (0) +/* The maximum length of a ring name. */ +#define _RING_NAMESIZE 32 +/* If set - ring is visible from different processes. + * Default is thread visible.*/ +#define _RING_SHM_PROC (1 << 2) +/* Ring size mask */ +#define _RING_SZ_MASK (unsigned)(0x0fffffff) + +typedef struct { + /* Rings tailq lock */ + odp_rwlock_t qlock; + odp_shm_t shm; +} global_data_t; + +static global_data_t *global; + +/* Initialize tailq_ring */ +static int _ring_global_init(void) +{ + odp_shm_t shm; + + /* Allocate globally shared memory */ + shm = odp_shm_reserve("_odp_ring_global", sizeof(global_data_t), + ODP_CACHE_LINE_SIZE, 0); + if (ODP_SHM_INVALID == shm) { + ODP_ERR("Shm reserve failed for pktio ring\n"); + return -1; + } + + global = odp_shm_addr(shm); + memset(global, 0, sizeof(global_data_t)); + global->shm = shm; + + return 0; +} + +static int _ring_global_term(void) +{ + if (odp_shm_free(global->shm)) { + ODP_ERR("Shm free failed for pktio ring\n"); + return -1; + } + return 0; +} + +/* create the ring */ +static ring_ptr_t * +_ring_create(const char *name, unsigned count, unsigned flags) +{ + char ring_name[_RING_NAMESIZE]; + ring_ptr_t *r; + size_t ring_size; + uint32_t shm_flag; + odp_shm_t shm; + + if (flags & _RING_SHM_PROC) + shm_flag = ODP_SHM_PROC | ODP_SHM_EXPORT; + else + shm_flag = 0; + if (odp_global_ro.shm_single_va) + shm_flag |= ODP_SHM_SINGLE_VA; + + /* count must be a power of 2 */ + if (!CHECK_IS_POWER2(count)) { + ODP_ERR("Requested size is invalid, must be power of 2," + "and do not exceed the size limit %u\n", + _RING_SZ_MASK); + __odp_errno = EINVAL; + return NULL; + } + + snprintf(ring_name, sizeof(ring_name), "%s", name); + ring_size = sizeof(ring_ptr_t) + count * sizeof(void *); + + /* reserve a memory zone for this ring.*/ + shm = odp_shm_reserve(ring_name, ring_size, ODP_CACHE_LINE_SIZE, + shm_flag); + + r = odp_shm_addr(shm); + if (r != NULL) { + /* init the ring structure */ + ring_ptr_init(r); + + } else { + __odp_errno = ENOMEM; + ODP_ERR("Cannot reserve memory\n"); + } + + return r; +} + +static int _ring_destroy(const char *name) +{ + odp_shm_t shm = odp_shm_lookup(name); + + if (shm != ODP_SHM_INVALID) + return odp_shm_free(shm); + + return 0; +} + +/** + * Return the number of entries in a ring. + */ +static unsigned _ring_count(ring_ptr_t *r, uint32_t mask) +{ + uint32_t prod_tail = odp_atomic_load_u32(&r->r.w_tail); + uint32_t cons_tail = odp_atomic_load_u32(&r->r.r_tail); + + return (prod_tail - cons_tail) & mask; +} + +/** + * Return the number of free entries in a ring. + */ +static unsigned _ring_free_count(ring_ptr_t *r, uint32_t mask) +{ + uint32_t prod_tail = odp_atomic_load_u32(&r->r.w_tail); + uint32_t cons_tail = odp_atomic_load_u32(&r->r.r_tail); + + return (cons_tail - prod_tail - 1) & mask; +} + typedef struct { /* TX */ struct { diff --git a/platform/linux-generic/pktio/ring.c b/platform/linux-generic/pktio/ring.c deleted file mode 100644 index 8b3d2b26e..000000000 --- a/platform/linux-generic/pktio/ring.c +++ /dev/null @@ -1,199 +0,0 @@ -/* Copyright (c) 2014-2018, Linaro Limited - * All rights reserved. - * - * SPDX-License-Identifier: BSD-3-Clause - */ - -/*- - * BSD LICENSE - * - * Copyright(c) 2010-2013 Intel Corporation. All rights reserved. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in - * the documentation and/or other materials provided with the - * distribution. - * * Neither the name of Intel Corporation nor the names of its - * contributors may be used to endorse or promote products derived - * from this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - */ - -/* - * Derived from FreeBSD's bufring.c - * - ************************************************************************** - * - * Copyright (c) 2007,2008 Kip Macy kmacy(a)freebsd.org - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * 1. Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * - * 2. The name of Kip Macy nor the names of other - * contributors may be used to endorse or promote products derived from - * this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" - * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE - * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR - * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS - * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN - * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE - * POSSIBILITY OF SUCH DAMAGE. - * - ***************************************************************************/ - -#include <odp_api.h> -#include <fcntl.h> -#include <stdio.h> -#include <string.h> -#include <stdbool.h> -#include <inttypes.h> -#include <odp_packet_io_ring_internal.h> -#include <odp_errno_define.h> -#include <odp_global_data.h> -#include <odp_align_internal.h> - -#include <odp_ring_ptr_internal.h> - -#include <odp/api/plat/cpu_inlines.h> - -typedef struct { - /* Rings tailq lock */ - odp_rwlock_t qlock; - odp_shm_t shm; -} global_data_t; - -static global_data_t *global; - -/* Initialize tailq_ring */ -int _ring_global_init(void) -{ odp_shm_t shm; - - /* Allocate globally shared memory */ - shm = odp_shm_reserve("_odp_ring_global", sizeof(global_data_t), - ODP_CACHE_LINE_SIZE, 0); - if (ODP_SHM_INVALID == shm) { - ODP_ERR("Shm reserve failed for pktio ring\n"); - return -1; - } - - global = odp_shm_addr(shm); - memset(global, 0, sizeof(global_data_t)); - global->shm = shm; - - return 0; -} - -int _ring_global_term(void) -{ - if (odp_shm_free(global->shm)) { - ODP_ERR("Shm free failed for pktio ring\n"); - return -1; - } - return 0; -} - -/* create the ring */ -ring_ptr_t * -_ring_create(const char *name, unsigned count, unsigned flags) -{ - char ring_name[_RING_NAMESIZE]; - ring_ptr_t *r; - size_t ring_size; - uint32_t shm_flag; - odp_shm_t shm; - - if (flags & _RING_SHM_PROC) - shm_flag = ODP_SHM_PROC | ODP_SHM_EXPORT; - else - shm_flag = 0; - if (odp_global_ro.shm_single_va) - shm_flag |= ODP_SHM_SINGLE_VA; - - /* count must be a power of 2 */ - if (!CHECK_IS_POWER2(count)) { - ODP_ERR("Requested size is invalid, must be power of 2," - "and do not exceed the size limit %u\n", - _RING_SZ_MASK); - __odp_errno = EINVAL; - return NULL; - } - - snprintf(ring_name, sizeof(ring_name), "%s", name); - ring_size = sizeof(ring_ptr_t) + count * sizeof(void *); - - /* reserve a memory zone for this ring.*/ - shm = odp_shm_reserve(ring_name, ring_size, ODP_CACHE_LINE_SIZE, - shm_flag); - - r = odp_shm_addr(shm); - if (r != NULL) { - /* init the ring structure */ - ring_ptr_init(r); - - } else { - __odp_errno = ENOMEM; - ODP_ERR("Cannot reserve memory\n"); - } - - return r; -} - -int _ring_destroy(const char *name) -{ - odp_shm_t shm = odp_shm_lookup(name); - - if (shm != ODP_SHM_INVALID) - return odp_shm_free(shm); - - return 0; -} - -/** - * Return the number of entries in a ring. - */ -unsigned _ring_count(ring_ptr_t *r, uint32_t mask) -{ - uint32_t prod_tail = odp_atomic_load_u32(&r->r.w_tail); - uint32_t cons_tail = odp_atomic_load_u32(&r->r.r_tail); - - return (prod_tail - cons_tail) & mask; -} - -/** - * Return the number of free entries in a ring. - */ -unsigned _ring_free_count(ring_ptr_t *r, uint32_t mask) -{ - uint32_t prod_tail = odp_atomic_load_u32(&r->r.w_tail); - uint32_t cons_tail = odp_atomic_load_u32(&r->r.r_tail); - - return (cons_tail - prod_tail - 1) & mask; -} commit 559aec0db739c624652facd3b337d0430cba8c61 Author: Matias Elo <matias.elo(a)nokia.com> Date: Thu Oct 24 15:51:47 2019 +0300 linux-gen: ipc: use standard ring_ptr_t implementation Use standard ODP internal ring_ptr_t rings as IPC rings. This enables removing the duplicate _ring_t implementation. Signed-off-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen(a)nokia.com> diff --git a/platform/linux-generic/include/odp_packet_io_ipc_internal.h b/platform/linux-generic/include/odp_packet_io_ipc_internal.h index 01410dab1..59c268cb9 100644 --- a/platform/linux-generic/include/odp_packet_io_ipc_internal.h +++ b/platform/linux-generic/include/odp_packet_io_ipc_internal.h @@ -20,6 +20,8 @@ /* number of odp buffers in odp ring queue */ #define PKTIO_IPC_ENTRIES 4096 +#define PKTIO_IPC_ENTRY_MASK (PKTIO_IPC_ENTRIES - 1) + /* that struct is exported to shared memory, so that processes can find * each other. */ diff --git a/platform/linux-generic/include/odp_packet_io_ring_internal.h b/platform/linux-generic/include/odp_packet_io_ring_internal.h index 6b4e06a42..22f1ae017 100644 --- a/platform/linux-generic/include/odp_packet_io_ring_internal.h +++ b/platform/linux-generic/include/odp_packet_io_ring_internal.h @@ -103,6 +103,8 @@ extern "C" { #include <sys/queue.h> #include <odp_debug_internal.h> +#include <odp_ring_ptr_internal.h> + enum _ring_queue_behavior { _RING_QUEUE_FIXED = 0, /**< Enq/Deq a fixed number of items from a ring */ @@ -158,8 +160,6 @@ typedef struct _ring { /* If set - ring is visible from different processes. * Default is thread visible.*/ #define _RING_SHM_PROC (1 << 2) - /* Do not link ring to linked list. */ -#define _RING_NO_LIST (1 << 3) /* Ring size mask */ #define _RING_SZ_MASK (unsigned)(0x0fffffff) @@ -190,8 +190,7 @@ typedef struct _ring { * - EEXIST - a memzone with the same name already exists * - ENOMEM - no appropriate memory area found in which to create memzone */ -_ring_t *_ring_create(const char *name, unsigned count, - unsigned flags); +ring_ptr_t *_ring_create(const char *name, unsigned count, unsigned flags); /** * Destroy the ring created with *name*. @@ -206,7 +205,7 @@ int _ring_destroy(const char *name); * * @param r A pointer to the ring structure. */ -void _ring_dump(const _ring_t *r); +void _ring_dump(ring_ptr_t *r, uint32_t mask); /** * Enqueue several objects on the ring (multi-producers safe). @@ -231,7 +230,7 @@ void _ring_dump(const _ring_t *r); * if behavior = ODPH_RING_QUEUE_VARIABLE * - n: Actual number of objects enqueued. */ -int ___ring_mp_do_enqueue(_ring_t *r, void * const *obj_table, +int ___ring_mp_do_enqueue(ring_ptr_t *r, void * const *obj_table, unsigned n, enum _ring_queue_behavior behavior); @@ -262,7 +261,7 @@ int ___ring_mp_do_enqueue(_ring_t *r, void * const *obj_table, * - n: Actual number of objects dequeued. */ -int ___ring_mc_do_dequeue(_ring_t *r, void **obj_table, +int ___ring_mc_do_dequeue(ring_ptr_t *r, void **obj_table, unsigned n, enum _ring_queue_behavior behavior); @@ -284,7 +283,7 @@ int ___ring_mc_do_dequeue(_ring_t *r, void **obj_table, * high water mark is exceeded. * - -ENOBUFS: Not enough room in the ring to enqueue, no object is enqueued. */ -int _ring_mp_enqueue_bulk(_ring_t *r, void * const *obj_table, +int _ring_mp_enqueue_bulk(ring_ptr_t *r, void * const *obj_table, unsigned n); /** @@ -304,7 +303,7 @@ int _ring_mp_enqueue_bulk(_ring_t *r, void * const *obj_table, * - -ENOENT: Not enough entries in the ring to dequeue; no object is * dequeued. */ -int _ring_mc_dequeue_bulk(_ring_t *r, void **obj_table, unsigned n); +int _ring_mc_dequeue_bulk(ring_ptr_t *r, void **obj_table, unsigned n); /** * Test if a ring is full. @@ -315,7 +314,7 @@ int _ring_mc_dequeue_bulk(_ring_t *r, void **obj_table, unsigned n); * - 1: The ring is full. * - 0: The ring is not full. */ -int _ring_full(const _ring_t *r); +int _ring_full(ring_ptr_t *r, uint32_t mask); /** * Test if a ring is empty. @@ -326,7 +325,7 @@ int _ring_full(const _ring_t *r); * - 1: The ring is empty. * - 0: The ring is not empty. */ -int _ring_empty(const _ring_t *r); +int _ring_empty(ring_ptr_t *r, uint32_t mask); /** * Return the number of entries in a ring. @@ -336,7 +335,7 @@ int _ring_empty(const _ring_t *r); * @return * The number of entries in the ring. */ -unsigned _ring_count(const _ring_t *r); +unsigned _ring_count(ring_ptr_t *r, uint32_t mask); /** * Return the number of free entries in a ring. @@ -346,14 +345,14 @@ unsigned _ring_count(const _ring_t *r); * @return * The number of free entries in the ring. */ -unsigned _ring_free_count(const _ring_t *r); +unsigned _ring_free_count(ring_ptr_t *r, uint32_t mask); /** * search ring by name * @param name ring name to search * @return pointer to ring otherwise NULL */ -_ring_t *_ring_lookup(const char *name); +ring_ptr_t *_ring_lookup(const char *name); /** * Enqueue several objects on the ring (multi-producers safe). @@ -370,7 +369,7 @@ _ring_t *_ring_lookup(const char *name); * @return * - n: Actual number of objects enqueued. */ -int _ring_mp_enqueue_burst(_ring_t *r, void * const *obj_table, +int _ring_mp_enqueue_burst(ring_ptr_t *r, void * const *obj_table, unsigned n); /** @@ -390,7 +389,7 @@ int _ring_mp_enqueue_burst(_ring_t *r, void * const *obj_table, * @return * - n: Actual number of objects dequeued, 0 if ring is empty */ -int _ring_mc_dequeue_burst(_ring_t *r, void **obj_table, unsigned n); +int _ring_mc_dequeue_burst(ring_ptr_t *r, void **obj_table, unsigned n); /** * dump the status of all rings on the console @@ -400,12 +399,12 @@ void _ring_list_dump(void); /** * Initialize ring tailq */ -int _ring_tailq_init(void); +int _ring_global_init(void); /** * Terminate ring tailq */ -int _ring_tailq_term(void); +int _ring_global_term(void); #ifdef __cplusplus } diff --git a/platform/linux-generic/pktio/ipc.c b/platform/linux-generic/pktio/ipc.c index 1090d760c..a7678f261 100644 --- a/platform/linux-generic/pktio/ipc.c +++ b/platform/linux-generic/pktio/ipc.c @@ -9,7 +9,7 @@ #include <odp_packet_io_internal.h> #include <odp/api/system_info.h> #include <odp_shm_internal.h> -#include <odp_shm_internal.h> +#include <odp_ring_ptr_internal.h> #include <sys/mman.h> #include <sys/stat.h> @@ -26,22 +26,23 @@ typedef struct { /* TX */ struct { - _ring_t *send; /**< ODP ring for IPC msg packets - indexes transmitted to shared - memory */ - _ring_t *free; /**< ODP ring for IPC msg packets - indexes already processed by remote - process */ + /* ODP ring for IPC msg packets indexes transmitted to shared + * memory */ + ring_ptr_t *send; + /* ODP ring for IPC msg packets indexes already processed by + * remote process */ + ring_ptr_t *free; } tx; /* RX */ struct { - _ring_t *recv; /**< ODP ring for IPC msg packets - indexes received from shared - memory (from remote process) */ - _ring_t *free; /**< odp ring for ipc msg packets - indexes already processed by - current process */ - _ring_t *cache; /**< local cache to keep packet order right */ + /* ODP ring for IPC msg packets indexes received from shared + * memory (from remote process) */ + ring_ptr_t *recv; + /* odp ring for ipc msg packets indexes already processed by + * current process */ + ring_ptr_t *free; + /* local cache to keep packet order right */ + ring_ptr_t *cache; } rx; /* slave */ void *pool_base; /**< Remote pool base addr */ void *pool_mdata_base; /**< Remote pool mdata base addr */ @@ -136,58 +137,66 @@ static int _ipc_init_master(pktio_entry_t *pktio_entry, */ snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_m_prod", dev); pkt_priv(pktio_entry)->tx.send = _ring_create(ipc_shm_name, - PKTIO_IPC_ENTRIES, - _RING_SHM_PROC | _RING_NO_LIST); + PKTIO_IPC_ENTRIES, + _RING_SHM_PROC); if (!pkt_priv(pktio_entry)->tx.send) { ODP_ERR("pid %d unable to create ipc ring %s name\n", getpid(), ipc_shm_name); return -1; } ODP_DBG("Created IPC ring: %s, count %d, free %d\n", - ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->tx.send), - _ring_free_count(pkt_priv(pktio_entry)->tx.send)); + ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->tx.send, + PKTIO_IPC_ENTRY_MASK), + _ring_free_count(pkt_priv(pktio_entry)->tx.send, + PKTIO_IPC_ENTRY_MASK)); /* generate name in shm like ipc_pktio_p for * already processed packets */ snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_m_cons", dev); pkt_priv(pktio_entry)->tx.free = _ring_create(ipc_shm_name, - PKTIO_IPC_ENTRIES, - _RING_SHM_PROC | _RING_NO_LIST); + PKTIO_IPC_ENTRIES, + _RING_SHM_PROC); if (!pkt_priv(pktio_entry)->tx.free) { ODP_ERR("pid %d unable to create ipc ring %s name\n", getpid(), ipc_shm_name); goto free_m_prod; } ODP_DBG("Created IPC ring: %s, count %d, free %d\n", - ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->tx.free), - _ring_free_count(pkt_priv(pktio_entry)->tx.free)); + ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->tx.free, + PKTIO_IPC_ENTRY_MASK), + _ring_free_count(pkt_priv(pktio_entry)->tx.free, + PKTIO_IPC_ENTRY_MASK)); snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_s_prod", dev); pkt_priv(pktio_entry)->rx.recv = _ring_create(ipc_shm_name, - PKTIO_IPC_ENTRIES, - _RING_SHM_PROC | _RING_NO_LIST); + PKTIO_IPC_ENTRIES, + _RING_SHM_PROC); if (!pkt_priv(pktio_entry)->rx.recv) { ODP_ERR("pid %d unable to create ipc ring %s name\n", getpid(), ipc_shm_name); goto free_m_cons; } ODP_DBG("Created IPC ring: %s, count %d, free %d\n", - ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->rx.recv), - _ring_free_count(pkt_priv(pktio_entry)->rx.recv)); + ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->rx.recv, + PKTIO_IPC_ENTRY_MASK), + _ring_free_count(pkt_priv(pktio_entry)->rx.recv, + PKTIO_IPC_ENTRY_MASK)); snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_s_cons", dev); pkt_priv(pktio_entry)->rx.free = _ring_create(ipc_shm_name, - PKTIO_IPC_ENTRIES, - _RING_SHM_PROC | _RING_NO_LIST); + PKTIO_IPC_ENTRIES, + _RING_SHM_PROC); if (!pkt_priv(pktio_entry)->rx.free) { ODP_ERR("pid %d unable to create ipc ring %s name\n", getpid(), ipc_shm_name); goto free_s_prod; } ODP_DBG("Created IPC ring: %s, count %d, free %d\n", - ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->rx.free), - _ring_free_count(pkt_priv(pktio_entry)->rx.free)); + ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->rx.free, + PKTIO_IPC_ENTRY_MASK), + _ring_free_count(pkt_priv(pktio_entry)->rx.free, + PKTIO_IPC_ENTRY_MASK)); /* Set up pool name for remote info */ pinfo = pkt_priv(pktio_entry)->pinfo; @@ -301,8 +310,10 @@ static int _ipc_slave_start(pktio_entry_t *pktio_entry) return -1; } ODP_DBG("Connected IPC ring: %s, count %d, free %d\n", - ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->rx.recv), - _ring_free_count(pkt_priv(pktio_entry)->rx.recv)); + ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->rx.recv, + PKTIO_IPC_ENTRY_MASK), + _ring_free_count(pkt_priv(pktio_entry)->rx.recv, + PKTIO_IPC_ENTRY_MASK)); snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_m_cons", dev); pkt_priv(pktio_entry)->rx.free = _ipc_shm_map(ipc_shm_name, pid); @@ -312,8 +323,10 @@ static int _ipc_slave_start(pktio_entry_t *pktio_entry) goto free_m_prod; } ODP_DBG("Connected IPC ring: %s, count %d, free %d\n", - ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->rx.free), - _ring_free_count(pkt_priv(pktio_entry)->rx.free)); + ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->rx.free, + PKTIO_IPC_ENTRY_MASK), + _ring_free_count(pkt_priv(pktio_entry)->rx.free, + PKTIO_IPC_ENTRY_MASK)); snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_s_prod", dev); pkt_priv(pktio_entry)->tx.send = _ipc_shm_map(ipc_shm_name, pid); @@ -323,8 +336,10 @@ static int _ipc_slave_start(pktio_entry_t *pktio_entry) goto free_m_cons; } ODP_DBG("Connected IPC ring: %s, count %d, free %d\n", - ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->tx.send), - _ring_free_count(pkt_priv(pktio_entry)->tx.send)); + ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->tx.send, + PKTIO_IPC_ENTRY_MASK), + _ring_free_count(pkt_priv(pktio_entry)->tx.send, + PKTIO_IPC_ENTRY_MASK)); snprintf(ipc_shm_name, sizeof(ipc_shm_name), "%s_s_cons", dev); pkt_priv(pktio_entry)->tx.free = _ipc_shm_map(ipc_shm_name, pid); @@ -334,8 +349,10 @@ static int _ipc_slave_start(pktio_entry_t *pktio_entry) goto free_s_prod; } ODP_DBG("Connected IPC ring: %s, count %d, free %d\n", - ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->tx.free), - _ring_free_count(pkt_priv(pktio_entry)->tx.free)); + ipc_shm_name, _ring_count(pkt_priv(pktio_entry)->tx.free, + PKTIO_IPC_ENTRY_MASK), + _ring_free_count(pkt_priv(pktio_entry)->tx.free, + PKTIO_IPC_ENTRY_MASK)); /* Get info about remote pool */ pinfo = pkt_priv(pktio_entry)->pinfo; @@ -374,7 +391,7 @@ static int ipc_pktio_open(odp_pktio_t id ODP_UNUSED, odp_pool_t pool) { int ret = -1; - int pid ODP_UNUSED; + int pid; struct pktio_info *pinfo; char name[ODP_POOL_NAME_LEN + sizeof("_info")]; char tail[ODP_POOL_NAME_LEN]; @@ -389,8 +406,7 @@ static int ipc_pktio_open(odp_pktio_t id ODP_UNUSED, odp_atomic_init_u32(&pkt_priv(pktio_entry)->ready, 0); pkt_priv(pktio_entry)->rx.cache = _ring_create("ipc_rx_cache", - PKTIO_IPC_ENTRIES, - _RING_NO_LIST); + PKTIO_IPC_ENTRIES, 0); if (!pkt_priv(pktio_entry)->rx.cache) return -1; @@ -440,7 +456,8 @@ static int ipc_pktio_open(odp_pktio_t id ODP_UNUSED, return ret; } -static void _ipc_free_ring_packets(pktio_entry_t *pktio_entry, _ring_t *r) +static void _ipc_free_ring_packets(pktio_entry_t *pktio_entry, ring_ptr_t *r, + uint32_t r_mask) { uintptr_t offsets[PKTIO_IPC_ENTRIES]; int ret; @@ -458,8 +475,8 @@ static void _ipc_free_ring_packets(pktio_entry_t *pktio_entry, _ring_t *r) rbuf_p = (void *)&offsets; while (1) { - ret = _ring_mc_dequeue_burst(r, rbuf_p, - PKTIO_IPC_ENTRIES); + ret = ring_ptr_deq_multi(r, r_mask, rbuf_p, + PKTIO_IPC_ENTRIES - 1); if (ret <= 0) break; for (i = 0; i < ret; i++) { @@ -479,12 +496,11 @@ static int ipc_pktio_recv_lockless(pktio_entry_t *pktio_entry, { int pkts = 0; int i; - _ring_t *r; - _ring_t *r_p; + ring_ptr_t *r; + ring_ptr_t *r_p; uintptr_t offsets[PKTIO_IPC_ENTRIES]; void **ipcbufs_p = (void *)&offsets[0]; uint32_t ready; - int pkts_ring; ready = odp_atomic_load_u32(&pkt_priv(pktio_entry)->ready); if (odp_unlikely(!ready)) { @@ -492,11 +508,12 @@ static int ipc_pktio_recv_lockless(pktio_entry_t *pktio_entry, return 0; } - _ipc_free_ring_packets(pktio_entry, pkt_priv(pktio_entry)->tx.free); + _ipc_free_ring_packets(pktio_entry, pkt_priv(pktio_entry)->tx.free, + PKTIO_IPC_ENTRY_MASK); /* rx from cache */ r = pkt_priv(pktio_entry)->rx.cache; - pkts = _ring_mc_dequeue_burst(r, ipcbufs_p, len); + pkts = ring_ptr_deq_multi(r, PKTIO_IPC_ENTRY_MASK, ipcbufs_p, len); if (odp_unlikely(pkts < 0)) ODP_ABORT("internal error dequeue\n"); @@ -504,7 +521,8 @@ static int ipc_pktio_recv_lockless(pktio_entry_t *pktio_entry, if (pkts == 0) { ipcbufs_p = (void *)&offsets[0]; r = pkt_priv(pktio_entry)->rx.recv; - pkts = _ring_mc_dequeue_burst(r, ipcbufs_p, len); + pkts = ring_ptr_deq_multi(r, PKTIO_IPC_ENTRY_MASK, ipcbufs_p, + len); if (odp_unlikely(pkts < 0)) ODP_ABORT("internal error dequeue\n"); } @@ -575,14 +593,11 @@ static int ipc_pktio_recv_lockless(pktio_entry_t *pktio_entry, if (pkts != i) { ipcbufs_p = (void *)&offsets[i]; r_p = pkt_priv(pktio_entry)->rx.cache; - pkts_ring = _ring_mp_enqueue_burst(r_p, ipcbufs_p, pkts - i); - - if (pkts_ring != (pkts - i)) - ODP_ABORT("bug to enqueue packets\n"); + ring_ptr_enq_multi(r_p, PKTIO_IPC_ENTRY_MASK, ipcbufs_p, + pkts - i); if (i == 0) return 0; - } /*num of actually received packets*/ @@ -591,28 +606,14 @@ static int ipc_pktio_recv_lockless(pktio_entry_t *pktio_entry, /* Now tell other process that we no longer need that buffers.*/ r_p = pkt_priv(pktio_entry)->rx.free; -repeat: - ipcbufs_p = (void *)&offsets[0]; - pkts_ring = _ring_mp_enqueue_burst(r_p, ipcbufs_p, pkts); - if (odp_unlikely(pkts_ring < 0)) - ODP_ABORT("ipc: odp_ring_mp_enqueue_bulk r_p fail\n"); + ring_ptr_enq_multi(r_p, PKTIO_IPC_ENTRY_MASK, ipcbufs_p, pkts); for (i = 0; i < pkts; i++) { IPC_ODP_DBG("%d/%d send to be free packet offset %x\n", i, pkts, offsets[i]); } - if (odp_unlikely(pkts != pkts_ring)) { - IPC_ODP_DBG("odp_ring_full: %d, odp_ring_count %d," - " _ring_free_count %d\n", - _ring_full(r_p), _ring_count(r_p), - _ring_free_count(r_p)); - ipcbufs_p = (void *)&offsets[pkts_ring - 1]; - pkts = pkts - pkts_ring; - goto repeat; - } - return pkts; } @@ -633,9 +634,8 @@ static int ipc_pktio_recv(pktio_entry_t *pktio_entry, int index ODP_UNUSED, static int ipc_pktio_send_lockless(pktio_entry_t *pktio_entry, const odp_packet_t pkt_table[], int num) { - _ring_t *r; + ring_ptr_t *r; void **rbuf_p; - int ret; int i; uint32_t ready = odp_atomic_load_u32(&pkt_priv(pktio_entry)->ready); pool_t *ipc_pool = pool_entry_from_hdl(pkt_priv(pktio_entry)->pool); @@ -646,7 +646,8 @@ static int ipc_pktio_send_lockless(pktio_entry_t *pktio_entry, if (odp_unlikely(!ready)) return 0; - _ipc_free_ring_packets(pktio_entry, pkt_priv(pktio_entry)->tx.free); + _ipc_free_ring_packets(pktio_entry, pkt_priv(pktio_entry)->tx.free, + PKTIO_IPC_ENTRY_MASK); /* Copy packets to shm shared pool if they are in different * pool, or if they are references (we can't share across IPC). @@ -703,17 +704,7 @@ static int ipc_pktio_send_lockless(pktio_entry_t *pktio_entry, /* Put packets to ring to be processed by other process. */ rbuf_p = (void *)&offsets[0]; r = pkt_priv(pktio_entry)->tx.send; - ret = _ring_mp_enqueue_burst(r, rbuf_p, num); - if (odp_unlikely(ret < 0)) { - ODP_ERR("pid %d odp_ring_mp_enqueue_bulk fail, ipc_slave %d, ret %d\n", - getpid(), - (PKTIO_TYPE_IPC_SLAVE == pkt_priv(pktio_entry)->type), - ret); - ODP_ERR("odp_ring_full: %d, odp_ring_count %d, _ring_free_count %d\n", - _ring_full(r), _ring_count(r), - _ring_free_count(r)); - ODP_ABORT("Unexpected!\n"); - } + ring_ptr_enq_multi(r, PKTIO_IPC_ENTRY_MASK, rbuf_p, num); return num; } @@ -762,28 +753,19 @@ static int ipc_start(pktio_entry_t *pktio_entry) static int ipc_stop(pktio_entry_t *pktio_entry) { - unsigned tx_send = 0, tx_free = 0; - odp_atomic_store_u32(&pkt_priv(pktio_entry)->ready, 0); if (pkt_priv(pktio_entry)->tx.send) _ipc_free_ring_packets(pktio_entry, - pkt_priv(pktio_entry)->tx.send); + pkt_priv(pktio_entry)->tx.send, + PKTIO_IPC_ENTRY_MASK); /* other process can transfer packets from one ring to * other, use delay here to free that packets. */ sleep(1); if (pkt_priv(pktio_entry)->tx.free) _ipc_free_ring_packets(pktio_entry, - pkt_priv(pktio_entry)->tx.free); - - if (pkt_priv(pktio_entry)->tx.send) - tx_send = _ring_count(pkt_priv(pktio_entry)->tx.send); - if (pkt_priv(pktio_entry)->tx.free) - tx_free = _ring_count(pkt_priv(pktio_entry)->tx.free); - if (tx_send | tx_free) { - ODP_DBG("IPC rings: tx send %d tx free %d\n", - tx_send, tx_free); - } + pkt_priv(pktio_entry)->tx.free, + PKTIO_IPC_ENTRY_MASK); return 0; } @@ -825,12 +807,12 @@ static int ipc_close(pktio_entry_t *pktio_entry) static int ipc_pktio_init_global(void) { ODP_DBG("PKTIO: initializing ipc interface.\n"); - return _ring_tailq_init(); + return _ring_global_init(); } static int ipc_pktio_term_global(void) { - return _ring_tailq_term(); + return _ring_global_term(); } const pktio_if_ops_t ipc_pktio_ops = { diff --git a/platform/linux-generic/pktio/ring.c b/platform/linux-generic/pktio/ring.c index 903e2c9f1..8b3d2b26e 100644 --- a/platform/linux-generic/pktio/ring.c +++ b/platform/linux-generic/pktio/ring.c @@ -78,13 +78,13 @@ #include <odp_packet_io_ring_internal.h> #include <odp_errno_define.h> #include <odp_global_data.h> +#include <odp_align_internal.h> -#include <odp/api/plat/cpu_inlines.h> +#include <odp_ring_ptr_internal.h> -#define RING_VAL_IS_POWER_2(x) ((((x) - 1) & (x)) == 0) +#include <odp/api/plat/cpu_inlines.h> typedef struct { - TAILQ_HEAD(, _ring) ring_list; /* Rings tailq lock */ odp_rwlock_t qlock; odp_shm_t shm; @@ -92,71 +92,8 @@ typedef struct { static global_data_t *global; -/* - * the enqueue of pointers on the ring. - */ -#define ENQUEUE_PTRS() do { \ - const uint32_t size = r->prod.size; \ - uint32_t idx = prod_head & mask; \ - if (odp_likely(idx + n < size)) { \ - for (i = 0; i < (n & ((~(unsigned)0x3))); i += 4, idx += 4) { \ - r->ring[idx] = obj_table[i]; \ - r->ring[idx + 1] = obj_table[i + 1]; \ - r->ring[idx + 2] = obj_table[i + 2]; \ - r->ring[idx + 3] = obj_table[i + 3]; \ - } \ - switch (n & 0x3) { \ - case 3: \ - r->ring[idx++] = obj_table[i++]; \ - /* fallthrough */ \ - case 2: \ - r->ring[idx++] = obj_table[i++]; \ - /* fallthrough */ \ - case 1: \ - r->ring[idx++] = obj_table[i++]; \ - } \ - } else { \ - for (i = 0; idx < size; i++, idx++)\ - r->ring[idx] = obj_table[i]; \ - for (idx = 0; i < n; i++, idx++) \ - r->ring[idx] = obj_table[i]; \ - } \ -} while (0) - -/* - * the actual copy of pointers on the ring to obj_table. - */ -#define DEQUEUE_PTRS() do { \ - uint32_t idx = cons_head & mask; \ - const uint32_t size = r->cons.size; \ - if (odp_likely(idx + n < size)) { \ - for (i = 0; i < (n & (~(unsigned)0x3)); i += 4, idx += 4) {\ - obj_table[i] = r->ring[idx]; \ - obj_table[i + 1] = r->ring[idx + 1]; \ - obj_table[i + 2] = r->ring[idx + 2]; \ - obj_table[i + 3] = r->ring[idx + 3]; \ - } \ - switch (n & 0x3) { \ - case 3: \ - obj_table[i++] = r->ring[idx++]; \ - /* fallthrough */ \ - case 2: \ - obj_table[i++] = r->ring[idx++]; \ - /* fallthrough */ \ - case 1: \ - obj_table[i++] = r->ring[idx++]; \ - } \ - } else { \ - for (i = 0; idx < size; i++, idx++) \ - obj_table[i] = r->ring[idx]; \ - for (idx = 0; i < n; i++, idx++) \ - obj_table[i] = r->ring[idx]; \ - } \ -} while (0) - - /* Initialize tailq_ring */ -int _ring_tailq_init(void) +int _ring_global_init(void) { odp_shm_t shm; /* Allocate globally shared memory */ @@ -171,14 +108,10 @@ int _ring_tailq_init(void) memset(global, 0, sizeof(global_data_t)); global->shm = shm; - TAILQ_INIT(&global->ring_list); - odp_rwlock_init(&global->qlock); - return 0; } -/* Terminate tailq_ring */ -int _ring_tailq_term(void) +int _ring_global_term(void) { if (odp_shm_free(global->shm)) { ODP_ERR("Shm free failed for pktio ring\n"); @@ -188,11 +121,11 @@ int _ring_tailq_term(void) } /* create the ring */ -_ring_t * +ring_ptr_t * _ring_create(const char *name, unsigned count, unsigned flags) { char ring_name[_RING_NAMESIZE]; - _ring_t *r; + ring_ptr_t *r; size_t ring_size; uint32_t shm_flag; odp_shm_t shm; @@ -205,7 +138,7 @@ _ring_create(const char *name, unsigned count, unsigned flags) shm_flag |= ODP_SHM_SINGLE_VA; /* count must be a power of 2 */ - if (!RING_VAL_IS_POWER_2(count) || (count > _RING_SZ_MASK)) { + if (!CHECK_IS_POWER2(count)) { ODP_ERR("Requested size is invalid, must be power of 2," "and do not exceed the size limit %u\n", _RING_SZ_MASK); @@ -214,36 +147,22 @@ _ring_create(const char *name, unsigned count, unsigned flags) } snprintf(ring_name, sizeof(ring_name), "%s", name); - ring_size = count * sizeof(void *) + sizeof(_ring_t); + ring_size = sizeof(ring_ptr_t) + count * sizeof(void *); - odp_rwlock_write_lock(&global->qlock); /* reserve a memory zone for this ring.*/ shm = odp_shm_reserve(ring_name, ring_size, ODP_CACHE_LINE_SIZE, shm_flag); r = odp_shm_addr(shm); - if (r != NULL) { /* init the ring structure */ - snprintf(r->name, sizeof(r->name), "%s", name); - r->flags = flags; - r->prod.size = count; - r->cons.size = count; - r->prod.mask = count - 1; - r->cons.mask = count - 1; - r->prod.head = 0; - r->cons.head = 0; - r->prod.tail = 0; - r->cons.tail = 0; + ring_ptr_init(r); - if (!(flags & _RING_NO_LIST)) - TAILQ_INSERT_TAIL(&global->ring_list, r, next); } else { __odp_errno = ENOMEM; ODP_ERR("Cannot reserve memory\n"); } - odp_rwlock_write_unlock(&global->qlock); return r; } @@ -251,265 +170,30 @@ int _ring_destroy(const char *name) { odp_shm_t shm = odp_shm_lookup(name); - if (shm != ODP_SHM_INVALID) { - _ring_t *r = odp_shm_addr(shm); - - odp_rwlock_write_lock(&global->qlock); - if (!(r->flags & _RING_NO_LIST)) - TAILQ_REMOVE(&global->ring_list, r, next); - odp_rwlock_write_unlock(&global->qlock); - + if (shm != ODP_SHM_INVALID) return odp_shm_free(shm); - } - return 0; -} - -/** - * Enqueue several objects on the ring (multi-producers safe). - */ -int ___ring_mp_do_enqueue(_ring_t *r, void * const *obj_table, - unsigned n, enum _ring_queue_behavior behavior) -{ - uint32_t prod_head, prod_next; - uint32_t cons_tail, free_entries; - const unsigned max = n; - int success; - unsigned i; - uint32_t mask = r->prod.mask; - - /* move prod.head atomically */ - do { - /* Reset n to the initial burst count */ - n = max; - - prod_head = __atomic_load_n(&r->prod.head, __ATOMIC_ACQUIRE); - cons_tail = __atomic_load_n(&r->cons.tail, __ATOMIC_ACQUIRE); - /* The subtraction is done between two unsigned 32bits value - * (the result is always modulo 32 bits even if we have - * prod_head > cons_tail). So 'free_entries' is always between 0 - * and size(ring)-1. */ - free_entries = (mask + cons_tail - prod_head); - - /* check that we have enough room in ring */ - if (odp_unlikely(n > free_entries)) { - if (behavior == _RING_QUEUE_FIXED) - return -ENOBUFS; - /* No free entry available */ - if (odp_unlikely(free_entries == 0)) - return 0; - - n = free_entries; - } - - prod_next = prod_head + n; - success = __atomic_compare_exchange_n(&r->prod.head, - &prod_head, - prod_next, - false/*strong*/, - __ATOMIC_ACQUIRE, - __ATOMIC_RELAXED); - } while (odp_unlikely(success == 0)); - - /* write entries in ring */ - ENQUEUE_PTRS(); - - /* - * If there are other enqueues in progress that preceded us, - * we need to wait for them to complete - */ - while (odp_unlikely(__atomic_load_n(&r->prod.tail, __ATOMIC_RELAXED) != - prod_head)) - odp_cpu_pause(); - - /* Release our entries and the memory they refer to */ - __atomic_store_n(&r->prod.tail, prod_next, __ATOMIC_RELEASE); - return (behavior == _RING_QUEUE_FIXED) ? 0 : n; -} - -/** - * Dequeue several objects from a ring (multi-consumers safe). - */ - -int ___ring_mc_do_dequeue(_ring_t *r, void **obj_table, - unsigned n, enum _ring_queue_behavior behavior) -{ - uint32_t cons_head, prod_tail; - uint32_t cons_next, entries; - const unsigned max = n; - int success; - unsigned i; - uint32_t mask = r->prod.mask; - - /* move cons.head atomically */ - do { - /* Restore n as it may change every loop */ - n = max; - - cons_head = __atomic_load_n(&r->cons.head, __ATOMIC_ACQUIRE); - prod_tail = __atomic_load_n(&r->prod.tail, __ATOMIC_ACQUIRE); - /* The subtraction is done between two unsigned 32bits value - * (the result is always modulo 32 bits even if we have - * cons_head > prod_tail). So 'entries' is always between 0 - * and size(ring)-1. */ - entries = (prod_tail - cons_head); - - /* Set the actual entries for dequeue */ - if (n > entries) { - if (behavior == _RING_QUEUE_FIXED) - return -ENOENT; - if (odp_unlikely(entries == 0)) - return 0; - - n = entries; - } - - cons_next = cons_head + n; - success = __atomic_compare_exchange_n(&r->cons.head, - &cons_head, - cons_next, - false/*strong*/, - __ATOMIC_ACQUIRE, - __ATOMIC_RELAXED); - } while (odp_unlikely(success == 0)); - - /* copy in table */ - DEQUEUE_PTRS(); - - /* - * If there are other dequeues in progress that preceded us, - * we need to wait for them to complete - */ - while (odp_unlikely(__atomic_load_n(&r->cons.tail, __ATOMIC_RELAXED) != - cons_head)) - odp_cpu_pause(); - - /* Release our entries and the memory they refer to */ - __atomic_store_n(&r->cons.tail, cons_next, __ATOMIC_RELEASE); - - return behavior == _RING_QUEUE_FIXED ? 0 : n; -} - -/** - * Enqueue several objects on the ring (multi-producers safe). - */ -int _ring_mp_enqueue_bulk(_ring_t *r, void * const *obj_table, - unsigned n) -{ - return ___ring_mp_do_enqueue(r, obj_table, n, - _RING_QUEUE_FIXED); -} - -/** - * Dequeue several objects from a ring (multi-consumers safe). - */ -int _ring_mc_dequeue_bulk(_ring_t *r, void **obj_table, unsigned n) -{ - return ___ring_mc_do_dequeue(r, obj_table, n, - _RING_QUEUE_FIXED); -} - -/** - * Test if a ring is full. - */ -int _ring_full(const _ring_t *r) -{ - uint32_t prod_tail = r->prod.tail; - uint32_t cons_tail = r->cons.tail; - return (((cons_tail - prod_tail - 1) & r->prod.mask) == 0); -} - -/** - * Test if a ring is empty. - */ -int _ring_empty(const _ring_t *r) -{ - uint32_t prod_tail = r->prod.tail; - uint32_t cons_tail = r->cons.tail; - - return !!(cons_tail == prod_tail); + return 0; } /** * Return the number of entries in a ring. */ -unsigned _ring_count(const _ring_t *r) +unsigned _ring_count(ring_ptr_t *r, uint32_t mask) { - uint32_t prod_tail = r->prod.tail; - uint32_t cons_tail = r->cons.tail; + uint32_t prod_tail = odp_atomic_load_u32(&r->r.w_tail); + uint32_t cons_tail = odp_atomic_load_u32(&r->r.r_tail); - return (prod_tail - cons_tail) & r->prod.mask; + return (prod_tail - cons_tail) & mask; } /** * Return the number of free entries in a ring. */ -unsigned _ring_free_count(const _ring_t *r) -{ - uint32_t prod_tail = r->prod.tail; - uint32_t cons_tail = r->cons.tail; - - return (cons_tail - prod_tail - 1) & r->prod.mask; -} - -/* dump the status of the ring on the console */ -void _ring_dump(const _ring_t *r) -{ - ODP_DBG("ring <%s>@%p\n", r->name, r); - ODP_DBG(" flags=%x\n", r->flags); - ODP_DBG(" size=%" PRIu32 "\n", r->prod.size); - ODP_DBG(" ct=%" PRIu32 "\n", r->cons.tail); - ODP_DBG(" ch=%" PRIu32 "\n", r->cons.head); - ODP_DBG(" pt=%" PRIu32 "\n", r->prod.tail); - ODP_DBG(" ph=%" PRIu32 "\n", r->prod.head); - ODP_DBG(" used=%u\n", _ring_count(r)); - ODP_DBG(" avail=%u\n", _ring_free_count(r)); -} - -/* dump the status of all rings on the console */ -void _ring_list_dump(void) -{ - const _ring_t *mp = NULL; - - odp_rwlock_read_lock(&global->qlock); - - TAILQ_FOREACH(mp, &global->ring_list, next) { - _ring_dump(mp); - } - - odp_rwlock_read_unlock(&global->qlock); -} - -/* search a ring from its name */ -_ring_t *_ring_lookup(const char *name) -{ - _ring_t *r; - - odp_rwlock_read_lock(&global->qlock); - TAILQ_FOREACH(r, &global->ring_list, next) { - if (strncmp(name, r->name, _RING_NAMESIZE) == 0) - break; - } - odp_rwlock_read_unlock(&global->qlock); - - return r; -} - -/** - * Enqueue several objects on the ring (multi-producers safe). - */ -int _ring_mp_enqueue_burst(_ring_t *r, void * const *obj_table, - unsigned n) +unsigned _ring_free_count(ring_ptr_t *r, uint32_t mask) { - return ___ring_mp_do_enqueue(r, obj_table, n, - _RING_QUEUE_VARIABLE); -} + uint32_t prod_tail = odp_atomic_load_u32(&r->r.w_tail); + uint32_t cons_tail = odp_atomic_load_u32(&r->r.r_tail); -/** - * Dequeue several objects from a ring (multi-consumers safe). - */ -int _ring_mc_dequeue_burst(_ring_t *r, void **obj_table, unsigned n) -{ - return ___ring_mc_do_dequeue(r, obj_table, n, - _RING_QUEUE_VARIABLE); + return (cons_tail - prod_tail - 1) & mask; } commit 720d655c9161007b561bf033a4fbe0b31ab7fb6a Author: Matias Elo <matias.elo(a)nokia.com> Date: Thu Oct 24 15:58:33 2019 +0300 linux-gen: ring: remove internal ring test Signed-off-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen(a)nokia.com> diff --git a/platform/linux-generic/m4/configure.m4 b/platform/linux-generic/m4/configure.m4 index 461dfc0fb..82fe6fe9c 100644 --- a/platform/linux-generic/m4/configure.m4 +++ b/platform/linux-generic/m4/configure.m4 @@ -38,6 +38,5 @@ AC_CONFIG_FILES([platform/linux-generic/Makefile platform/linux-generic/test/Makefile platform/linux-generic/test/validation/api/shmem/Makefile platform/linux-generic/test/validation/api/pktio/Makefile - platform/linux-generic/test/pktio_ipc/Makefile - platform/linux-generic/test/ring/Makefile]) + platform/linux-generic/test/pktio_ipc/Makefile]) ]) diff --git a/platform/linux-generic/test/Makefile.am b/platform/linux-generic/test/Makefile.am index d3193b5e9..870336fd2 100644 --- a/platform/linux-generic/test/Makefile.am +++ b/platform/linux-generic/test/Makefile.am @@ -23,8 +23,7 @@ test_SCRIPTS = $(dist_check_SCRIPTS) SUBDIRS += validation/api/pktio\ validation/api/shmem\ - pktio_ipc\ - ring + pktio_ipc if HAVE_PCAP TESTS += validation/api/pktio/pktio_run_pcap.sh diff --git a/platform/linux-generic/test/ring/.gitignore b/platform/linux-generic/test/ring/.gitignore deleted file mode 100644 index 7341a340c..000000000 --- a/platform/linux-generic/test/ring/.gitignore +++ /dev/null @@ -1 +0,0 @@ -ring_main diff --git a/platform/linux-generic/test/ring/Makefile.am b/platform/linux-generic/test/ring/Makefile.am deleted file mode 100644 index 999beed4f..000000000 --- a/platform/linux-generic/test/ring/Makefile.am +++ /dev/null @@ -1,38 +0,0 @@ -# ring test uses internal symbols from libodp-linux which are not available -# when linking test with libodp-linux.so -if STATIC_APPS - -include $(top_srcdir)/test/Makefile.inc - -test_PROGRAMS = ring_main -ring_main_SOURCES = \ - ring_main.c \ - ring_suites.c ring_suites.h \ - ring_basic.c ring_stress.c - -TESTS = ring_main$(EXEEXT) - -PRELDADD += $(LIBCUNIT_COMMON) - -AM_CPPFLAGS += -I$(top_srcdir)/platform/linux-generic/include - -AM_CFLAGS += $(LIBCONFIG_CFLAGS) - -TESTNAME = linux-generic-ring - -TESTENV = tests-$(TESTNAME).env - -test_DATA = $(TESTENV) - -DISTCLEANFILES = $(TESTENV) -.PHONY: $(TESTENV) -$(TESTENV): - echo "TESTS=\"$(TESTS)\"" > $@ - echo "$(TESTS_ENVIRONMENT)" >> $@ - echo "$(LOG_COMPILER)" >> $@ - -if test_installdir -installcheck-local: - $(DESTDIR)/$(testdir)/run-test.sh $(TESTNAME) -endif -endif diff --git a/platform/linux-generic/test/ring/ring_basic.c b/platform/linux-generic/test/ring/ring_basic.c deleted file mode 100644 index dc76fd6b3..000000000 --- a/platform/linux-generic/test/ring/ring_basic.c +++ /dev/null @@ -1,280 +0,0 @@ -/* Copyright (c) 2016-2018, Linaro Limited - * All rights reserved. - * - * SPDX-License-Identifier: BSD-3-Clause - */ - -/** - * @file - * - * ODP ring basic test - */ - -#include <stdlib.h> -#include <stdio.h> -#include <string.h> - -#include <odp_cunit_common.h> -#include <odp_packet_io_ring_internal.h> -#include <odp_errno_define.h> - -#include <odp/helper/odph_api.h> - -#include "ring_suites.h" - -/* labor functions declaration */ -static void __do_basic_burst(_ring_t *r); -static void __do_basic_bulk(_ring_t *r); - -/* dummy object pointers for enqueue and dequeue testing */ -static void **test_enq_data; -static void **test_deq_data; - -/* create multiple thread test ring */ -static const char *mt_ring_name = "MT basic ring"; -static _ring_t *mt_ring; - -int ring_test_basic_start(void) -{ - int i = 0; - - /* alloc dummy object pointers for enqueue testing */ - test_enq_data = malloc(RING_SIZE * 2 * sizeof(void *)); - if (NULL == test_enq_data) { - ODPH_ERR("failed to allocate basic test enqeue data\n"); - return -1; - } - - for (i = 0; i < RING_SIZE * 2; i++) - test_enq_data[i] = (void *)(unsigned long)i; - - /* alloc dummy object pointers for dequeue testing */ - test_deq_data = malloc(RING_SIZE * 2 * sizeof(void *)); - if (NULL == test_deq_data) { - ODPH_ERR("failed to allocate basic test dequeue data\n"); - free(test_enq_data); test_enq_data = NULL; - return -1; - } - - memset(test_deq_data, 0, RING_SIZE * 2 * sizeof(void *)); - return 0; -} - -int ring_test_basic_end(void) -{ - _ring_destroy(mt_ring_name); - - free(test_enq_data); - free(test_deq_data); - return 0; -} - -/* basic test cases */ -void ring_test_basic_create(void) -{ - /* prove illegal size shall fail */ - mt_ring = _ring_create(mt_ring_name, ILLEGAL_SIZE, 0); - CU_ASSERT(NULL == mt_ring); - CU_ASSERT(EINVAL == __odp_errno); - - /* create ring for multiple thread usage scenario */ - mt_ring = _ring_create(mt_ring_name, RING_SIZE, - _RING_SHM_PROC); - - CU_ASSERT(NULL != mt_ring); - CU_ASSERT(_ring_lookup(mt_ring_name) == mt_ring); -} - -void ring_test_basic_burst(void) -{ - __do_basic_burst(mt_ring); -} - -void ring_test_basic_bulk(void) -{ - __do_basic_bulk(mt_ring); -} - -/* labor functions definition */ -static void __do_basic_burst(_ring_t *r) -{ - int result = 0; - unsigned int count = 0; - void * const *source = test_enq_data; - void * const *dest = test_deq_data; - void **enq = NULL, **deq = NULL; - - enq = test_enq_data; deq = test_deq_data; - - /* ring is empty */ - CU_ASSERT(1 == _ring_empty(r)); - - /* enqueue 1 object */ - result = _ring_mp_enqueue_burst(r, enq, 1); - enq += 1; - CU_ASSERT(1 == (result & _RING_SZ_MASK)); - - /* enqueue 2 objects */ - result = _ring_mp_enqueue_burst(r, enq, 2); - enq += 2; - CU_ASSERT(2 == (result & _RING_SZ_MASK)); - - /* enqueue HALF_BULK objects */ - result = _ring_mp_enqueue_burst(r, enq, HALF_BULK); - enq += HALF_BULK; - CU_ASSERT(HALF_BULK == (result & _RING_SZ_MASK)); - - /* ring is neither empty nor full */ - CU_ASSERT(0 == _ring_full(r)); - CU_ASSERT(0 == _ring_empty(r)); - - /* _ring_count() equals enqueued */ - count = (1 + 2 + HALF_BULK); - CU_ASSERT(count == _ring_count(r)); - /* _ring_free_count() equals rooms left */ - count = (RING_SIZE - 1) - count; - CU_ASSERT(count == _ring_free_count(r)); - - /* exceed the size, enquene as many as possible */ - result = _ring_mp_enqueue_burst(r, enq, HALF_BULK); - enq += count; - CU_ASSERT(count == (result & _RING_SZ_MASK)); - CU_ASSERT(1 == _ring_full(r)); - - /* dequeue 1 object */ - result = _ring_mc_dequeue_burst(r, deq, 1); - deq += 1; - CU_ASSERT(1 == (result & _RING_SZ_MASK)); - - /* dequeue 2 objects */ - result = _ring_mc_dequeue_burst(r, deq, 2); - deq += 2; - CU_ASSERT(2 == (result & _RING_SZ_MASK)); - - /* dequeue HALF_BULK objects */ - result = _ring_mc_dequeue_burst(r, deq, HALF_BULK); - deq += HALF_BULK; - CU_ASSERT(HALF_BULK == (result & _RING_SZ_MASK)); - - /* _ring_free_count() equals dequeued */ - count = (1 + 2 + HALF_BULK); - CU_ASSERT(count == _ring_free_count(r)); - /* _ring_count() equals remained left */ - count = (RING_SIZE - 1) - count; - CU_ASSERT(count == _ring_count(r)); - - /* underrun the size, dequeue as many as possible */ - result = _ring_mc_dequeue_burst(r, deq, HALF_BULK); - deq += count; - CU_ASSERT(count == (result & _RING_SZ_MASK)); - CU_ASSERT(1 == _ring_empty(r)); - - /* check data */ - CU_ASSERT(0 == memcmp(source, dest, deq - dest)); - - /* reset dequeue data */ - memset(test_deq_data, 0, RING_SIZE * 2 * sizeof(void *)); -} - -/* incomplete ring API set: strange! - * complement _ring_enqueue/dequeue_bulk to improve coverage - */ -static inline int __ring_enqueue_bulk( - _ring_t *r, void * const *objects, unsigned bulk) -{ - return _ring_mp_enqueue_bulk(r, objects, bulk); -} - -static inline int __ring_dequeue_bulk( - _ring_t *r, void **objects, unsigned bulk) -{ - return _ring_mc_dequeue_bulk(r, objects, bulk); -} - -static void __do_basic_bulk(_ring_t *r) -{ - int result = 0; - unsigned int count = 0; - void * const *source = test_enq_data; - void * const *dest = test_deq_data; - void **enq = NULL, **deq = NULL; - - enq = test_enq_data; deq = test_deq_data; - - /* ring is empty */ - CU_ASSERT(1 == _ring_empty(r)); - - /* enqueue 1 object */ - result = __ring_enqueue_bulk(r, enq, 1); - enq += 1; - CU_ASSERT(0 == result); - - /* enqueue 2 objects */ - result = __ring_enqueue_bulk(r, enq, 2); - enq += 2; - CU_ASSERT(0 == result); - - /* enqueue HALF_BULK objects */ - result = __ring_enqueue_bulk(r, enq, HALF_BULK); - enq += HALF_BULK; - CU_ASSERT(0 == result); - - /* ring is neither empty nor full */ - CU_ASSERT(0 == _ring_full(r)); - CU_ASSERT(0 == _ring_empty(r)); - - /* _ring_count() equals enqueued */ - count = (1 + 2 + HALF_BULK); - CU_ASSERT(count == _ring_count(r)); - /* _ring_free_count() equals rooms left */ - count = (RING_SIZE - 1) - count; - CU_ASSERT(count == _ring_free_count(r)); - - /* exceed the size, enquene shall fail with -ENOBUFS */ - result = __ring_enqueue_bulk(r, enq, HALF_BULK); - CU_ASSERT(-ENOBUFS == result); - - /* fullful the ring */ - result = __ring_enqueue_bulk(r, enq, count); - enq += count; - CU_ASSERT(0 == result); - CU_ASSERT(1 == _ring_full(r)); - - /* dequeue 1 object */ - result = __ring_dequeue_bulk(r, deq, 1); - deq += 1; - CU_ASSERT(0 == result); - - /* dequeue 2 objects */ - result = __ring_dequeue_bulk(r, deq, 2); - deq += 2; - CU_ASSERT(0 == result); - - /* dequeue HALF_BULK objects */ - result = __ring_dequeue_bulk(r, deq, HALF_BULK); - deq += HALF_BULK; - CU_ASSERT(0 == result); - - /* _ring_free_count() equals dequeued */ - count = (1 + 2 + HALF_BULK); - CU_ASSERT(count == _ring_free_count(r)); - /* _ring_count() equals remained left */ - count = (RING_SIZE - 1) - count; - CU_ASSERT(count == _ring_count(r)); - - /* underrun the size, dequeue shall fail with -ENOENT */ - result = __ring_dequeue_bulk(r, deq, HALF_BULK); - CU_ASSERT(-ENOENT == result); - - /* empty the queue */ - result = __ring_dequeue_bulk(r, deq, count); - deq += count; - CU_ASSERT(0 == result); - CU_ASSERT(1 == _ring_empty(r)); - - /* check data */ - CU_ASSERT(0 == memcmp(source, dest, deq - dest)); - - /* reset dequeue data */ - memset(test_deq_data, 0, RING_SIZE * 2 * sizeof(void *)); -} diff --git a/platform/linux-generic/test/ring/ring_main.c b/platform/linux-generic/test/ring/ring_main.c deleted file mode 100644 index 8d0f07527..000000000 --- a/platform/linux-generic/test/ring/ring_main.c +++ /dev/null @@ -1,12 +0,0 @@ -/* Copyright (c) 2016-2018, Linaro Limited - * All rights reserved. - * - * SPDX-License-Identifier: BSD-3-Clause - */ - -#include "ring_suites.h" - -int main(int argc, char *argv[]) -{ - return ring_suites_main(argc, argv); -} diff --git a/platform/linux-generic/test/ring/ring_stress.c b/platform/linux-generic/test/ring/ring_stress.c deleted file mode 100644 index b0b996789..000000000 --- a/platform/linux-generic/test/ring/ring_stress.c +++ /dev/null @@ -1,270 +0,0 @@ -/* Copyright (c) 2016-2018, Linaro Limited - * All rights reserved. - * - * SPDX-License-Identifier: BSD-3-Clause - */ - -/** - * @file - * - * ODP ring stress test - */ - -#ifndef _GNU_SOURCE -#define _GNU_SOURCE -#endif - -#include <stdlib.h> -#include <stdio.h> -#include <string.h> -#include <unistd.h> - -#include <odp_api.h> -#include <odp/helper/odph_api.h> -#include <odp_packet_io_ring_internal.h> -#include <odp_cunit_common.h> - -#include "ring_suites.h" - -/* There's even number of producer and consumer threads and each thread does - * this many successful enq or deq operations */ -#define NUM_BULK_OP ((RING_SIZE / PIECE_BULK) * 100) - -/* - * Note : make sure that both enqueue and dequeue - * operation starts at same time so to avoid data corruption - * Its because atomic lock will protect only indexes, but if order of - * read or write operation incorrect then data mismatch will happen - * So its resposibility of application develop to take care of order of - * data read or write. - */ -typedef enum { - STRESS_1_1_PRODUCER_CONSUMER, - STRESS_1_N_PRODUCER_CONSUMER, - STRESS_N_1_PRODUCER_CONSUMER, - STRESS_N_M_PRODUCER_CONSUMER -} stress_case_t; - -#define GLOBAL_SHM_NAME "RingGlobalShm" - -/* worker function declarations */ -static int stress_worker(void *_data); - -/* global name for later look up in workers' context */ -static const char *ring_name = "stress_ring"; - -typedef struct { - odp_shm_t shm; - /* Barrier to run threads at the same time */ - odp_barrier_t barrier; - /* - * Since cunit framework cannot work with multi-threading, ask workers - * to save their results for delayed assertion after thread collection. - */ - int worker_results[MAX_WORKERS]; -} global_shared_mem_t; - -static global_shared_mem_t *global_mem; - -int ring_test_stress_start(void) -{ - _ring_t *r_stress = NULL; - odp_shm_t shm; - - shm = odp_shm_reserve(GLOBAL_SHM_NAME, sizeof(global_shared_mem_t), 64, - ODP_SHM_SW_ONLY); - if (shm == ODP_SHM_INVALID) { - fprintf(stderr, "Unable reserve memory for global_shm\n"); - return -1; - } - - global_mem = odp_shm_addr(shm); - memset(global_mem, 0, sizeof(global_shared_mem_t)); - global_mem->shm = shm; - - /* multiple thread usage scenario, thread or process sharable */ - r_stress = _ring_create(ring_name, RING_SIZE, _RING_SHM_PROC); - if (r_stress == NULL) { - ODPH_ERR("create ring failed for stress.\n"); - return -1; - } - - return 0; -} - -int ring_test_stress_end(void) -{ - if (odp_shm_free(global_mem->shm)) { - fprintf(stderr, "error: odp_shm_free() failed.\n"); - return -1; - } - - return 0; -} - -void ring_test_stress_1_1_producer_consumer(void) -{ - int i = 0; - odp_cpumask_t cpus; - pthrd_arg worker_param; - - /* reset results for delayed assertion */ - memset(global_mem->worker_results, 0, - sizeof(global_mem->worker_results)); - - /* request 2 threads to run 1:1 stress */ - worker_param.numthrds = odp_cpumask_default_worker(&cpus, 2); - worker_param.testcase = STRESS_1_1_PRODUCER_CONSUMER; - - /* not failure, insufficient resource */ - if (worker_param.numthrds < 2) { - ODPH_ERR("insufficient cpu for 1:1 " - "producer/consumer stress.\n"); - return; - } - - odp_barrier_init(&global_mem->barrier, 2); - - /* kick the workers */ - odp_cunit_thread_create(stress_worker, &worker_param); - - /* collect the results */ - odp_cunit_thread_exit(&worker_param); - - /* delayed assertion due to cunit limitation */ - for (i = 0; i < worker_param.numthrds; i++) - CU_ASSERT(0 == global_mem->worker_results[i]); -} - -void ring_test_stress_N_M_producer_consumer(void) -{ - int i = 0; - odp_cpumask_t cpus; - pthrd_arg worker_param; - - /* reset results for delayed assertion */ - memset(global_mem->worker_results, 0, - sizeof(global_mem->worker_results)); - - /* request MAX_WORKERS threads to run N:M stress */ - worker_param.numthrds = - odp_cpumask_default_worker(&cpus, MAX_WORKERS); - worker_param.testcase = STRESS_N_M_PRODUCER_CONSUMER; - - /* not failure, insufficient resource */ - if (worker_param.numthrds < 3) { - ODPH_ERR("insufficient cpu for N:M " - "producer/consumer stress.\n"); - return; - } - - /* force even number of threads */ - if (worker_param.numthrds & 0x1) - worker_param.numthrds -= 1; - - odp_barrier_init(&global_mem->barrier, worker_param.numthrds); - - /* kick the workers */ - odp_cunit_thread_create(stress_worker, &worker_param); - - /* collect the results */ - odp_cunit_thread_exit(&worker_param); - - /* delayed assertion due to cunit limitation */ - for (i = 0; i < worker_param.numthrds; i++) - CU_ASSERT(0 == global_mem->worker_results[i]); -} - -void ring_test_stress_1_N_producer_consumer(void) -{ -} - -void ring_test_stress_N_1_producer_consumer(void) -{ -} - -void ring_test_stress_ring_list_dump(void) -{ - /* improve code coverage */ - _ring_list_dump(); -} - -/* worker function for multiple producer instances */ -static int do_producer(_ring_t *r) -{ - void *enq[PIECE_BULK]; - int i; - int num = NUM_BULK_OP; - - /* data pattern to be evaluated later in consumer */ - for (i = 0; i < PIECE_BULK; i++) - enq[i] = (void *)(uintptr_t)i; - - while (num) - if (_ring_mp_enqueue_bulk(r, enq, PIECE_BULK) == 0) - num--; - - return 0; -} - -/* worker function for multiple consumer instances */ -static int do_consumer(_ring_t *r) -{ - void *deq[PIECE_BULK]; - int i; - int num = NUM_BULK_OP; - - while (num) { - if (_ring_mc_dequeue_bulk(r, deq, PIECE_BULK) == 0) { - num--; - - /* evaluate the data pattern */ - for (i = 0; i < PIECE_BULK; i++) - CU_ASSERT(deq[i] == (void *)(uintptr_t)i); - } - } - - return 0; -} - -static int stress_worker(void *_data) -{ - pthrd_arg *worker_param = (pthrd_arg *)_data; - _ring_t *r_stress = NULL; - int *result = NULL; - int worker_id = odp_thread_id(); - - /* save the worker result for delayed assertion */ - result = &global_mem->worker_results[(worker_id % - worker_param->numthrds)]; - - /* verify ring lookup in worker context */ - r_stress = _ring_lookup(ring_name); - if (NULL == r_stress) { - ODPH_ERR("ring lookup %s not found\n", ring_name); - return (*result = -1); - } - - odp_barrier_wait(&global_mem->barrier); - - switch (worker_param->testcase) { - case STRESS_1_1_PRODUCER_CONSUMER: - case STRESS_N_M_PRODUCER_CONSUMER: - /* interleaved producer/consumer */ - if (0 == (worker_id % 2)) - *result = do_producer(r_stress); - else if (1 == (worker_id % 2)) - *result = do_consumer(r_stress); - break; - case STRESS_1_N_PRODUCER_CONSUMER: - case STRESS_N_1_PRODUCER_CONSUMER: - default: - ODPH_ERR("invalid or not-implemented stress type (%d)\n", - worker_param->testcase); - break; - } - - odp_barrier_wait(&global_mem->barrier); - - return 0; -} diff --git a/platform/linux-generic/test/ring/ring_suites.c b/platform/linux-generic/test/ring/ring_suites.c deleted file mode 100644 index a6b3376ce..000000000 --- a/platform/linux-generic/test/ring/ring_suites.c +++ /dev/null @@ -1,74 +0,0 @@ -/* Copyright (c) 2016-2018, Linaro Limited - * All rights reserved. - * - * SPDX-License-Identifier: BSD-3-Clause - */ - -#include <stdlib.h> -#include <stdio.h> -#include <string.h> - -#include <odp_api.h> -#include <odp/helper/odph_api.h> - -#include <odp_cunit_common.h> -#include <odp_packet_io_ring_internal.h> - -#include "ring_suites.h" - -static int ring_suites_init(odp_instance_t *inst) -{ - if (0 != odp_init_global(inst, NULL, NULL)) { - ODPH_ERR("error: odp_init_global() failed.\n"); - return -1; - } - if (0 != odp_init_local(*inst, ODP_THREAD_CONTROL)) { - ODPH_ERR("error: odp_init_local() failed.\n"); - return -1; - } - - _ring_tailq_init(); - return 0; -} - -static odp_testinfo_t ring_suite_basic[] = { - ODP_TEST_INFO(ring_test_basic_create), - ODP_TEST_INFO(ring_test_basic_burst), - ODP_TEST_INFO(ring_test_basic_bulk), - ODP_TEST_INFO_NULL, -}; - -static odp_testinfo_t ring_suite_stress[] = { - ODP_TEST_INFO(ring_test_stress_1_1_producer_consumer), - ODP_TEST_INFO(ring_test_stress_1_N_producer_consumer), - ODP_TEST_INFO(ring_test_stress_N_1_producer_consumer), - ODP_TEST_INFO(ring_test_stress_N_M_producer_consumer), - ODP_TEST_INFO(ring_test_stress_ring_list_dump), - ODP_TEST_INFO_NULL, -}; - -static odp_suiteinfo_t ring_suites[] = { - {"ring basic", ring_test_basic_start, - ring_test_basic_end, ring_suite_basic}, - {"ring stress", ring_test_stress_start, - ring_test_stress_end, ring_suite_stress}, - ODP_SUITE_INFO_NULL -}; - -int ring_suites_main(int argc, char *argv[]) -{ - int ret; - - /* let helper collect its own arguments (e.g. --odph_proc) */ - if (odp_cunit_parse_options(argc, argv)) - return -1; - - odp_cunit_register_global_init(ring_suites_init); - - ret = odp_cunit_register(ring_suites); - - if (ret == 0) - ret = odp_cunit_run(); - - return ret; -} diff --git a/platform/linux-generic/test/ring/ring_suites.h b/platform/linux-generic/test/ring/ring_suites.h deleted file mode 100644 index d56ab7845..000000000 --- a/platform/linux-generic/test/ring/ring_suites.h +++ /dev/null @@ -1,33 +0,0 @@ -/* Copyright (c) 2016-2018, Linaro Limited - * All rights reserved. - * - * SPDX-License-Identifier: BSD-3-Clause - */ - -#define RING_SIZE 4096 -#define PIECE_BULK 32 - -#define HALF_BULK (RING_SIZE >> 1) -#define ILLEGAL_SIZE (RING_SIZE | 0x3) - -/* test suite start and stop */ -int ring_test_basic_start(void); -int ring_test_basic_end(void); - -/* basic test cases */ -void ring_test_basic_create(void); -void ring_test_basic_burst(void); -void ring_test_basic_bulk(void); - -/* test suite start and stop */ -int ring_test_stress_start(void); -int ring_test_stress_end(void); - -/* stress test cases */ -void ring_test_stress_1_1_producer_consumer(void); -void ring_test_stress_1_N_producer_consumer(void); -void ring_test_stress_N_1_producer_consumer(void); -void ring_test_stress_N_M_producer_consumer(void); -void ring_test_stress_ring_list_dump(void); - -int ring_suites_main(int argc, char *argv[]); ----------------------------------------------------------------------- Summary of changes: platform/linux-generic/Makefile.am | 3 - .../linux-generic/include/odp_buffer_internal.h | 4 - .../linux-generic/include/odp_packet_io_internal.h | 1 - .../include/odp_packet_io_ipc_internal.h | 47 -- .../include/odp_packet_io_ring_internal.h | 414 ---------------- platform/linux-generic/m4/configure.m4 | 3 +- platform/linux-generic/odp_packet_io.c | 1 - platform/linux-generic/pktio/ipc.c | 536 ++++++++++++--------- platform/linux-generic/pktio/ring.c | 515 -------------------- platform/linux-generic/test/Makefile.am | 3 +- platform/linux-generic/test/ring/.gitignore | 1 - platform/linux-generic/test/ring/Makefile.am | 38 -- platform/linux-generic/test/ring/ring_basic.c | 280 ----------- platform/linux-generic/test/ring/ring_main.c | 12 - platform/linux-generic/test/ring/ring_stress.c | 270 ----------- platform/linux-generic/test/ring/ring_suites.c | 74 --- platform/linux-generic/test/ring/ring_suites.h | 33 -- 17 files changed, 321 insertions(+), 1914 deletions(-) delete mode 100644 platform/linux-generic/include/odp_packet_io_ipc_internal.h delete mode 100644 platform/linux-generic/include/odp_packet_io_ring_internal.h delete mode 100644 platform/linux-generic/pktio/ring.c delete mode 100644 platform/linux-generic/test/ring/.gitignore delete mode 100644 platform/linux-generic/test/ring/Makefile.am delete mode 100644 platform/linux-generic/test/ring/ring_basic.c delete mode 100644 platform/linux-generic/test/ring/ring_main.c delete mode 100644 platform/linux-generic/test/ring/ring_stress.c delete mode 100644 platform/linux-generic/test/ring/ring_suites.c delete mode 100644 platform/linux-generic/test/ring/ring_suites.h hooks/post-receive --

6 years, 10 months

[COMMIT] branch master updated. v1.21.0.0-299-g1a8b32c75

by git＠git-us.linaro.org

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 1a8b32c758894b0d0a71ebbd86efb77fd9eb334c (commit) via d3dd92949a71783f54530e06e5afa15ec9f507c2 (commit) via 61af8e3ac83e75e66a404edbbac9718ce99d95c0 (commit) via da423931fe6b6f4532bc05a71c7323bf28f7be2a (commit) via 5ae87b8706ae7d85a1c14a63359c737c6988abce (commit) from 5930362eaf96ea150987c4e4e69b17f09a8a8f3a (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 1a8b32c758894b0d0a71ebbd86efb77fd9eb334c Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Mon Nov 4 14:11:56 2019 +0200 example: cls: add support for UDP/TCP ports Added support for UDP/TCP port PMR terms, and others with a 16 bit value. Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> diff --git a/example/classifier/odp_classifier.c b/example/classifier/odp_classifier.c index 3af418704..f0cab8c05 100644 --- a/example/classifier/odp_classifier.c +++ b/example/classifier/odp_classifier.c @@ -37,7 +37,7 @@ /** @def MAX_PMR_COUNT * @brief Maximum number of Classification Policy */ -#define MAX_PMR_COUNT 8 +#define MAX_PMR_COUNT 32 /** @def DISPLAY_STRING_LEN * @brief Length of string used to display term value @@ -757,9 +757,27 @@ static int convert_str_to_pmr_enum(char *token, odp_cls_pmr_term_t *term) if (strcasecmp(token, "ODP_PMR_ETHTYPE_0") == 0) { *term = ODP_PMR_ETHTYPE_0; return 0; + } else if (strcasecmp(token, "ODP_PMR_ETHTYPE_X") == 0) { + *term = ODP_PMR_ETHTYPE_X; + return 0; } else if (strcasecmp(token, "ODP_PMR_VLAN_ID_0") == 0) { *term = ODP_PMR_VLAN_ID_0; return 0; + } else if (strcasecmp(token, "ODP_PMR_VLAN_ID_X") == 0) { + *term = ODP_PMR_VLAN_ID_X; + return 0; + } else if (strcasecmp(token, "ODP_PMR_UDP_DPORT") == 0) { + *term = ODP_PMR_UDP_DPORT; + return 0; + } else if (strcasecmp(token, "ODP_PMR_TCP_DPORT") == 0) { + *term = ODP_PMR_TCP_DPORT; + return 0; + } else if (strcasecmp(token, "ODP_PMR_UDP_SPORT") == 0) { + *term = ODP_PMR_UDP_SPORT; + return 0; + } else if (strcasecmp(token, "ODP_PMR_TCP_SPORT") == 0) { + *term = ODP_PMR_TCP_SPORT; + return 0; } else if (strcasecmp(token, "ODP_PMR_SIP_ADDR") == 0) { *term = ODP_PMR_SIP_ADDR; return 0; @@ -792,7 +810,8 @@ static int parse_pmr_policy(appl_args_t *appl_args, char *optarg) /* last array index is needed for default queue */ if (policy_count >= MAX_PMR_COUNT - 1) { - ODPH_ERR("Maximum allowed PMR reached\n"); + ODPH_ERR("Too many policies. Max count is %i.\n", + MAX_PMR_COUNT - 1); return -1; } @@ -814,10 +833,23 @@ static int parse_pmr_policy(appl_args_t *appl_args, char *optarg) /* PMR value */ switch (term) { case ODP_PMR_ETHTYPE_0: - /* :<type>:<mask> */ /* Fall through */ + case ODP_PMR_ETHTYPE_X: + /* Fall through */ + /* :<type>:<mask> */ case ODP_PMR_VLAN_ID_0: + /* Fall through */ + case ODP_PMR_VLAN_ID_X: + /* Fall through */ /* :<vlan_id>:<mask> */ + case ODP_PMR_UDP_DPORT: + /* Fall through */ + case ODP_PMR_TCP_DPORT: + /* Fall through */ + case ODP_PMR_UDP_SPORT: + /* Fall through */ + case ODP_PMR_TCP_SPORT: + /* :<port>:<mask> */ token = strtok(NULL, ":"); strncpy(stats[policy_count].value, token, DISPLAY_STRING_LEN - 1); @@ -951,7 +983,7 @@ static int parse_args(int argc, char *argv[], appl_args_t *appl_args) appl_args->cpu_count = 1; /* Use one worker by default */ appl_args->verbose = 0; - while (1) { + while (ret == 0) { opt = getopt_long(argc, argv, shortopts, longopts, &long_index); @@ -963,8 +995,10 @@ static int parse_args(int argc, char *argv[], appl_args_t *appl_args) appl_args->cpu_count = atoi(optarg); break; case 'p': - if (0 > parse_pmr_policy(appl_args, optarg)) - continue; + if (parse_pmr_policy(appl_args, optarg)) { + ret = -1; + break; + } policy = 1; break; case 't': commit d3dd92949a71783f54530e06e5afa15ec9f507c2 Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Fri Nov 1 14:14:45 2019 +0200 validation: cls: add parallel PMR test Test serial and parallel PMR and CoS: - from default CoS to dest IPv4 CoS - from dest IPv4 CoS to a parallel UDP CoS per destination port number Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> diff --git a/test/validation/api/classification/odp_classification_test_pmr.c b/test/validation/api/classification/odp_classification_test_pmr.c index 62f8a7425..7cbeca586 100644 --- a/test/validation/api/classification/odp_classification_test_pmr.c +++ b/test/validation/api/classification/odp_classification_test_pmr.c @@ -2367,6 +2367,11 @@ static void classification_test_pmr_serial(void) test_pmr_series(1); } +static void classification_test_pmr_parallel(void) +{ + test_pmr_series(4); +} + static void classification_test_pmr_term_custom_frame(void) { test_pmr_term_custom(0); @@ -2511,6 +2516,8 @@ odp_testinfo_t classification_suite_pmr[] = { check_capa_custom_l3), ODP_TEST_INFO_CONDITIONAL(classification_test_pmr_serial, check_capa_pmr_series), + ODP_TEST_INFO_CONDITIONAL(classification_test_pmr_parallel, + check_capa_pmr_series), ODP_TEST_INFO(classification_test_pktin_classifier_flag), ODP_TEST_INFO(classification_test_pmr_term_tcp_dport_multi), ODP_TEST_INFO_NULL, commit 61af8e3ac83e75e66a404edbbac9718ce99d95c0 Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Fri Nov 1 14:10:19 2019 +0200 validation: cls: add serial PMR test Test series of PMR and CoS: - from default CoS to dest IPv4 CoS - from dest IPv4 CoS to dest UDP CoS Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> diff --git a/test/validation/api/classification/odp_classification_test_pmr.c b/test/validation/api/classification/odp_classification_test_pmr.c index d4a3110e3..62f8a7425 100644 --- a/test/validation/api/classification/odp_classification_test_pmr.c +++ b/test/validation/api/classification/odp_classification_test_pmr.c @@ -2168,6 +2168,205 @@ static void test_pmr_term_custom(int custom_l3) odp_pktio_close(pktio); } +/* + * Test a series of PMR rules and CoS. When num_udp is 1, test is serial + * from IP CoS to UDP CoS. When num_udp is larger than 1, a set of parallel + * UDP CoS are tested. + * + * dst IP dst UDP[0 ... 3] + * default_cos -> cos_ip -> cos_udp[0 ... 3] + */ +static void test_pmr_series(const int num_udp) +{ + odp_packet_t pkt; + uint32_t seqno; + int i, retval; + cls_packet_info_t pkt_info; + odp_pktio_t pktio; + odp_pool_t pool; + odp_queue_t default_queue; + odp_pool_t default_pool; + odp_cos_t default_cos; + odp_queue_t retqueue; + odp_pmr_t pmr_ip; + odp_queue_t queue_ip; + odp_cos_t cos_ip; + uint32_t dst_addr; + uint32_t dst_addr_be, ip_mask_be; + uint32_t dst_mask; + odp_pmr_param_t pmr_param; + odp_cls_cos_param_t cls_param; + odph_ethhdr_t *eth; + odph_ipv4hdr_t *ip; + odph_udphdr_t *udp; + odp_cos_t cos_udp[num_udp]; + odp_queue_t queue_udp[num_udp]; + odp_pmr_t pmr_udp[num_udp]; + uint16_t dst_port = 1000; + + pktio = create_pktio(ODP_QUEUE_TYPE_SCHED, pkt_pool, true); + retval = start_pktio(pktio); + CU_ASSERT(retval == 0); + + configure_default_cos(pktio, &default_cos, + &default_queue, &default_pool); + + pool = pool_create("pmr_series"); + CU_ASSERT_FATAL(pool != ODP_POOL_INVALID); + + /* Dest IP address */ + queue_ip = queue_create("queue_ip", true); + CU_ASSERT_FATAL(queue_ip != ODP_QUEUE_INVALID); + + odp_cls_cos_param_init(&cls_param); + cls_param.pool = pool; + cls_param.queue = queue_ip; + cls_param.drop_policy = ODP_COS_DROP_POOL; + + cos_ip = odp_cls_cos_create("cos_ip", &cls_param); + CU_ASSERT_FATAL(cos_ip != ODP_COS_INVALID); + + parse_ipv4_string("10.0.9.99/32", &dst_addr, &dst_mask); + dst_addr_be = odp_cpu_to_be_32(dst_addr); + ip_mask_be = odp_cpu_to_be_32(dst_mask); + + odp_cls_pmr_param_init(&pmr_param); + pmr_param.term = ODP_PMR_DIP_ADDR; + pmr_param.match.value = &dst_addr_be; + pmr_param.match.mask = &ip_mask_be; + pmr_param.val_sz = sizeof(dst_addr_be); + pmr_param.offset = 0; + + pmr_ip = odp_cls_pmr_create(&pmr_param, 1, default_cos, cos_ip); + CU_ASSERT_FATAL(pmr_ip != ODP_PMR_INVALID); + + /* Dest UDP port */ + for (i = 0; i < num_udp; i++) { + uint16_t dst_port_be = odp_cpu_to_be_16(dst_port + i); + uint16_t port_mask_be = odp_cpu_to_be_16(0xffff); + char name[] = "udp_0"; + + name[4] += i; + queue_udp[i] = queue_create(name, true); + CU_ASSERT_FATAL(queue_udp[i] != ODP_QUEUE_INVALID); + + odp_cls_cos_param_init(&cls_param); + cls_param.pool = pool; + cls_param.queue = queue_udp[i]; + cls_param.drop_policy = ODP_COS_DROP_POOL; + + cos_udp[i] = odp_cls_cos_create(name, &cls_param); + CU_ASSERT_FATAL(cos_udp[i] != ODP_COS_INVALID); + + odp_cls_pmr_param_init(&pmr_param); + pmr_param.term = ODP_PMR_UDP_DPORT; + pmr_param.match.value = &dst_port_be; + pmr_param.match.mask = &port_mask_be; + pmr_param.val_sz = 2; + pmr_param.offset = 0; + + pmr_udp[i] = odp_cls_pmr_create(&pmr_param, 1, + cos_ip, cos_udp[i]); + CU_ASSERT_FATAL(pmr_udp[i] != ODP_PMR_INVALID); + } + + /* Matching TCP/IP packet */ + pkt_info = default_pkt_info; + pkt_info.udp = false; + + pkt = create_packet(pkt_info); + CU_ASSERT_FATAL(pkt != ODP_PACKET_INVALID); + + eth = (odph_ethhdr_t *)odp_packet_l2_ptr(pkt, NULL); + ip = (odph_ipv4hdr_t *)odp_packet_l3_ptr(pkt, NULL); + + odp_pktio_mac_addr(pktio, eth->src.addr, ODPH_ETHADDR_LEN); + odp_pktio_mac_addr(pktio, eth->dst.addr, ODPH_ETHADDR_LEN); + ip->dst_addr = dst_addr_be; + + seqno = cls_pkt_get_seq(pkt); + CU_ASSERT(seqno != TEST_SEQ_INVALID); + + enqueue_pktio_interface(pkt, pktio); + + pkt = receive_packet(&retqueue, ODP_TIME_SEC_IN_NS); + CU_ASSERT_FATAL(pkt != ODP_PACKET_INVALID); + CU_ASSERT(seqno == cls_pkt_get_seq(pkt)); + CU_ASSERT(retqueue == queue_ip); + odp_packet_free(pkt); + + /* Matching UDP/IP packets */ + pkt_info = default_pkt_info; + pkt_info.udp = true; + + for (i = 0; i < num_udp; i++) { + pkt = create_packet(pkt_info); + CU_ASSERT_FATAL(pkt != ODP_PACKET_INVALID); + + eth = (odph_ethhdr_t *)odp_packet_l2_ptr(pkt, NULL); + ip = (odph_ipv4hdr_t *)odp_packet_l3_ptr(pkt, NULL); + udp = (odph_udphdr_t *)odp_packet_l4_ptr(pkt, NULL); + + odp_pktio_mac_addr(pktio, eth->src.addr, ODPH_ETHADDR_LEN); + odp_pktio_mac_addr(pktio, eth->dst.addr, ODPH_ETHADDR_LEN); + ip->dst_addr = dst_addr_be; + udp->dst_port = odp_cpu_to_be_16(dst_port + i); + + seqno = cls_pkt_get_seq(pkt); + CU_ASSERT(seqno != TEST_SEQ_INVALID); + + enqueue_pktio_interface(pkt, pktio); + + pkt = receive_packet(&retqueue, ODP_TIME_SEC_IN_NS); + CU_ASSERT_FATAL(pkt != ODP_PACKET_INVALID); + CU_ASSERT(seqno == cls_pkt_get_seq(pkt)); + CU_ASSERT(retqueue == queue_udp[i]); + odp_packet_free(pkt); + } + + /* Other packets delivered to default queue */ + pkt = create_packet(default_pkt_info); + CU_ASSERT_FATAL(pkt != ODP_PACKET_INVALID); + seqno = cls_pkt_get_seq(pkt); + CU_ASSERT(seqno != TEST_SEQ_INVALID); + eth = (odph_ethhdr_t *)odp_packet_l2_ptr(pkt, NULL); + odp_pktio_mac_addr(pktio, eth->src.addr, ODPH_ETHADDR_LEN); + odp_pktio_mac_addr(pktio, eth->dst.addr, ODPH_ETHADDR_LEN); + + enqueue_pktio_interface(pkt, pktio); + + pkt = receive_packet(&retqueue, ODP_TIME_SEC_IN_NS); + CU_ASSERT_FATAL(pkt != ODP_PACKET_INVALID); + CU_ASSERT(seqno == cls_pkt_get_seq(pkt)); + CU_ASSERT(retqueue == default_queue); + odp_packet_free(pkt); + + for (i = 0; i < num_udp; i++) { + odp_cos_destroy(cos_udp[i]); + odp_cls_pmr_destroy(pmr_udp[i]); + } + + odp_cos_destroy(cos_ip); + odp_cls_pmr_destroy(pmr_ip); + + odp_cos_destroy(default_cos); + stop_pktio(pktio); + odp_pool_destroy(default_pool); + odp_pool_destroy(pool); + + for (i = 0; i < num_udp; i++) + odp_queue_destroy(queue_udp[i]); + + odp_queue_destroy(queue_ip); + odp_queue_destroy(default_queue); + odp_pktio_close(pktio); +} + +static void classification_test_pmr_serial(void) +{ + test_pmr_series(1); +} + static void classification_test_pmr_term_custom_frame(void) { test_pmr_term_custom(0); @@ -2263,6 +2462,16 @@ static int check_capa_custom_l3(void) return cls_capa.supported_terms.bit.custom_l3; } +static int check_capa_pmr_series(void) +{ + uint64_t support; + + support = cls_capa.supported_terms.bit.dip_addr && + cls_capa.supported_terms.bit.udp_dport; + + return support; +} + odp_testinfo_t classification_suite_pmr[] = { ODP_TEST_INFO_CONDITIONAL(classification_test_pmr_term_tcp_dport, check_capa_tcp_dport), @@ -2300,6 +2509,8 @@ odp_testinfo_t classification_suite_pmr[] = { check_capa_custom_frame), ODP_TEST_INFO_CONDITIONAL(classification_test_pmr_term_custom_l3, check_capa_custom_l3), + ODP_TEST_INFO_CONDITIONAL(classification_test_pmr_serial, + check_capa_pmr_series), ODP_TEST_INFO(classification_test_pktin_classifier_flag), ODP_TEST_INFO(classification_test_pmr_term_tcp_dport_multi), ODP_TEST_INFO_NULL, commit da423931fe6b6f4532bc05a71c7323bf28f7be2a Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Fri Nov 1 16:17:01 2019 +0200 linux-gen: cls: fix PMR match recursion Parallel PMR rules (leafs in classification tree) were not correctly checked. Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> diff --git a/platform/linux-generic/odp_classification.c b/platform/linux-generic/odp_classification.c index a803b52b2..23069b662 100644 --- a/platform/linux-generic/odp_classification.c +++ b/platform/linux-generic/odp_classification.c @@ -1275,10 +1275,10 @@ static int verify_pmr(pmr_t *pmr, const uint8_t *pkt_addr, } if (pmr_failure) - return false; + return 0; } odp_atomic_inc_u32(&pmr->s.count); - return true; + return 1; } /* @@ -1286,14 +1286,10 @@ static int verify_pmr(pmr_t *pmr, const uint8_t *pkt_addr, * This function gets called recursively to check the chained PMR Term value * with the packet. */ -static -cos_t *match_pmr_cos(cos_t *cos, const uint8_t *pkt_addr, pmr_t *pmr, - odp_packet_hdr_t *hdr) +static cos_t *match_pmr_cos(cos_t *cos, const uint8_t *pkt_addr, pmr_t *pmr, + odp_packet_hdr_t *hdr) { - cos_t *retcos; - uint32_t i; - - retcos = NULL; + uint32_t i, num_rule; if (cos == NULL || pmr == NULL) return NULL; @@ -1302,19 +1298,29 @@ cos_t *match_pmr_cos(cos_t *cos, const uint8_t *pkt_addr, pmr_t *pmr, return NULL; if (verify_pmr(pmr, pkt_addr, hdr)) { - /** This gets called recursively to check all the PMRs in - * a PMR chain */ - if (0 == odp_atomic_load_u32(&cos->s.num_rule)) + /* This gets called recursively. First matching leaf or branch + * is returned. */ + num_rule = odp_atomic_load_u32(&cos->s.num_rule); + + /* No more rules. This is the best match. */ + if (num_rule == 0) return cos; - for (i = 0; i < odp_atomic_load_u32(&cos->s.num_rule); i++) { - retcos = match_pmr_cos(cos->s.linked_cos[i], pkt_addr, - cos->s.pmr[i], hdr); - if (!retcos) - return cos; + for (i = 0; i < num_rule; i++) { + cos_t *retcos = match_pmr_cos(cos->s.linked_cos[i], + pkt_addr, cos->s.pmr[i], + hdr); + + /* Found a matching leaf */ + if (retcos) + return retcos; } + + /* Current CoS was the best match */ + return cos; } - return retcos; + + return NULL; } int pktio_classifier_init(pktio_entry_t *entry) commit 5ae87b8706ae7d85a1c14a63359c737c6988abce Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Fri Nov 1 11:31:19 2019 +0200 linux-gen: cls: enable custom L3 protocols with CUSTOM_L3 Input flag for L3 is set only for known L3 protocols. Allow custom L3 protocols by checking L2 flag and L3 offset only. Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> diff --git a/platform/linux-generic/odp_classification.c b/platform/linux-generic/odp_classification.c index 1b5337ff6..a803b52b2 100644 --- a/platform/linux-generic/odp_classification.c +++ b/platform/linux-generic/odp_classification.c @@ -1087,7 +1087,8 @@ static inline int verify_pmr_custom_l3(const uint8_t *pkt_addr, ODP_ASSERT(val_sz <= MAX_PMR_TERM_SIZE); - if (!pkt_hdr->p.input_flags.l3) + if (pkt_hdr->p.input_flags.l2 == 0 || + l3_offset == ODP_PACKET_OFFSET_INVALID) return 0; if (packet_len(pkt_hdr) <= offset + val_sz) ----------------------------------------------------------------------- Summary of changes: example/classifier/odp_classifier.c | 46 ++++- platform/linux-generic/odp_classification.c | 45 +++-- .../classification/odp_classification_test_pmr.c | 218 +++++++++++++++++++++ 3 files changed, 284 insertions(+), 25 deletions(-) hooks/post-receive --

6 years, 10 months

[COMMIT] branch master updated. v1.21.0.0-294-g5930362ea

by git＠git-us.linaro.org

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 5930362eaf96ea150987c4e4e69b17f09a8a8f3a (commit) from 810bc2ee9b5285e1692a6b0c7457ca503f381e1e (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 5930362eaf96ea150987c4e4e69b17f09a8a8f3a Author: Matias Elo <matias.elo(a)nokia.com> Date: Fri Nov 1 15:22:59 2019 +0200 linux-gen: shm: improve memory allocation error messages Include page type (normal/huge) and filename in error messages. Signed-off-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen(a)nokia.com> diff --git a/platform/linux-generic/odp_ishm.c b/platform/linux-generic/odp_ishm.c index cd483d146..41f3979bf 100644 --- a/platform/linux-generic/odp_ishm.c +++ b/platform/linux-generic/odp_ishm.c @@ -681,8 +681,8 @@ static int create_file(int block_index, huge_flag_t huge, uint64_t len, fd = open(filename, oflag, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH); if (fd < 0) { if (huge != HUGE) - ODP_ERR("open failed for %s: %s.\n", - filename, strerror(errno)); + ODP_ERR("Normal page open failed: file=%s, " + "err=\"%s\"\n", filename, strerror(errno)); return -1; } @@ -694,8 +694,9 @@ static int create_file(int block_index, huge_flag_t huge, uint64_t len, } if (ret == -1) { - ODP_ERR("memory allocation failed: fd=%d, err=%s.\n", - fd, strerror(errno)); + ODP_ERR("%s memory allocation failed: fd=%d, file=%s, " + "err=\"%s\"\n", (huge == HUGE) ? "Huge page" : + "Normal page", fd, filename, strerror(errno)); close(fd); unlink(filename); return -1; ----------------------------------------------------------------------- Summary of changes: platform/linux-generic/odp_ishm.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) hooks/post-receive --

6 years, 10 months

[COMMIT] branch master updated. v1.21.0.0-293-g810bc2ee9

by git＠git-us.linaro.org

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 810bc2ee9b5285e1692a6b0c7457ca503f381e1e (commit) via bd2aea3186fd4e2d3ba6ec91f2a6ba72f0443941 (commit) via b904314ec95e4e680a739a5f98b6fd814b9a6fef (commit) via 2376fa5b4b5f5b7b9d2c11cb08438f11883bfe50 (commit) via 55ea0298a7cf917b466d2154d79d00fd86f3c936 (commit) via d688a269562ff9efc2ff03411c385a2bd1228d2c (commit) via 13b478eaa76ee8461341b114d447835e3906748d (commit) via 4c4145f8c9c73665c9f834f8ca639b9f0525e30f (commit) via 4c58ce0cd530c59907164f2fa7ae78800c494151 (commit) via 65891255eec4884e941aece9808ad684155feac8 (commit) via d4eb0481e1dd40a0d0b70486cdf8f9b676862b2f (commit) via f15da1c0ee13dd0e342944dccdd9014c16d4d5be (commit) via 6f855d843b1d87013908998a050e16c435b89e07 (commit) via 35a6b2e5c369e81196f6c288085990eaaf83f360 (commit) via 0e2ee113fcc0e7383d9e465c11a8495a65f7a171 (commit) via 0cfdb2c5a6b3afe0e4b0eee2b3749a0cfca3917d (commit) via b7a8fd47362a3853db8cc3ea56fe1d268defa574 (commit) via 0df7a38e2b9be4ad72528d43551e56ac4d152b96 (commit) via 139b28399d3bd1f53624908c9fd99c1f14283cdd (commit) from 86745e999aa2bf644fd72775df55042289e07b2b (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 810bc2ee9b5285e1692a6b0c7457ca503f381e1e Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Mon Oct 28 14:49:26 2019 +0200 api: increment version to 1.23 Increment API version number to reflect API changes: * cls: PMR values for protocol fields are defined in big endian * cls: ODP_PMR_CUSTOM_L3 term has been added * coding style: use 'unsigned int' instead of 'unsigned' Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Stanislaw Kardach <skardach(a)marvell.com> diff --git a/configure.ac b/configure.ac index 81123cbc6..4cff42913 100644 --- a/configure.ac +++ b/configure.ac @@ -3,7 +3,7 @@ AC_PREREQ([2.5]) # ODP API version ########################################################################## m4_define([odpapi_generation_version], [1]) -m4_define([odpapi_major_version], [22]) +m4_define([odpapi_major_version], [23]) m4_define([odpapi_minor_version], [0]) m4_define([odpapi_point_version], [0]) m4_define([odpapi_version], commit bd2aea3186fd4e2d3ba6ec91f2a6ba72f0443941 Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Mon Oct 28 15:33:05 2019 +0200 api: clean up coding style warnings Clean checkpatch warnings on include/odp/api/spec files. Current checkpatch version warns about 'unsigned' type name usage. Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Stanislaw Kardach <skardach(a)marvell.com> diff --git a/include/odp/api/spec/classification.h b/include/odp/api/spec/classification.h index eadcaa2d8..1db85e8b7 100644 --- a/include/odp/api/spec/classification.h +++ b/include/odp/api/spec/classification.h @@ -175,17 +175,17 @@ typedef struct odp_cls_capability_t { odp_cls_pmr_terms_t supported_terms; /** Maximum number of PMR terms */ - unsigned max_pmr_terms; + unsigned int max_pmr_terms; /** Number of PMR terms available for use now */ - unsigned available_pmr_terms; + unsigned int available_pmr_terms; /** Maximum number of CoS supported */ - unsigned max_cos; + unsigned int max_cos; /** Maximun number of queue supported per CoS * if the value is 1, then hashing is not supported*/ - unsigned max_hash_queues; + unsigned int max_hash_queues; /** Protocol header combination supported for Hashing */ odp_pktin_hash_proto_t hash_protocols; diff --git a/include/odp/api/spec/packet_io.h b/include/odp/api/spec/packet_io.h index 5d5ca2bc4..cddacecc5 100644 --- a/include/odp/api/spec/packet_io.h +++ b/include/odp/api/spec/packet_io.h @@ -201,7 +201,7 @@ typedef struct odp_pktin_queue_param_t { * More than one input queues require flow hashing configured. * The maximum value is defined by pktio capability 'max_input_queues'. * Queue type is defined by the input mode. The default value is 1. */ - unsigned num_queues; + unsigned int num_queues; /** Queue parameters * @@ -243,7 +243,7 @@ typedef struct odp_pktout_queue_param_t { /** Number of output queues to be created. The value must be between * 1 and interface capability. The default value is 1. */ - unsigned num_queues; + unsigned int num_queues; } odp_pktout_queue_param_t; @@ -551,10 +551,10 @@ typedef union odp_pktio_set_op_t { */ typedef struct odp_pktio_capability_t { /** Maximum number of input queues */ - unsigned max_input_queues; + unsigned int max_input_queues; /** Maximum number of output queues */ - unsigned max_output_queues; + unsigned int max_output_queues; /** Supported pktio configuration options */ odp_pktio_config_t config; @@ -650,7 +650,7 @@ int odp_pktio_capability(odp_pktio_t pktio, odp_pktio_capability_t *capa); * * @return Maximum packet IO interface index */ -unsigned odp_pktio_max_index(void); +unsigned int odp_pktio_max_index(void); /** * Configure packet IO interface options @@ -947,8 +947,8 @@ int odp_pktin_recv_tmo(odp_pktin_queue_t queue, odp_packet_t packets[], * @return Number of packets received * @retval <0 on failure */ -int odp_pktin_recv_mq_tmo(const odp_pktin_queue_t queues[], unsigned num_q, - unsigned *from, odp_packet_t packets[], int num, +int odp_pktin_recv_mq_tmo(const odp_pktin_queue_t queues[], unsigned int num_q, + unsigned int *from, odp_packet_t packets[], int num, uint64_t wait); /** diff --git a/include/odp/api/spec/pool.h b/include/odp/api/spec/pool.h index d4cf088f1..69e99605c 100644 --- a/include/odp/api/spec/pool.h +++ b/include/odp/api/spec/pool.h @@ -48,12 +48,12 @@ extern "C" { */ typedef struct odp_pool_capability_t { /** Maximum number of pools of any type */ - unsigned max_pools; + unsigned int max_pools; /** Buffer pool capabilities */ struct { /** Maximum number of buffer pools */ - unsigned max_pools; + unsigned int max_pools; /** Maximum buffer data alignment in bytes */ uint32_t max_align; @@ -74,7 +74,7 @@ typedef struct odp_pool_capability_t { /** Packet pool capabilities */ struct { /** Maximum number of packet pools */ - unsigned max_pools; + unsigned int max_pools; /** Maximum packet data length in bytes * @@ -147,7 +147,7 @@ typedef struct odp_pool_capability_t { /** Timeout pool capabilities */ struct { /** Maximum number of timeout pools */ - unsigned max_pools; + unsigned int max_pools; /** Maximum number of timeout events in a pool * diff --git a/include/odp/api/spec/shared_memory.h b/include/odp/api/spec/shared_memory.h index 0882c4a1c..425e6c77a 100644 --- a/include/odp/api/spec/shared_memory.h +++ b/include/odp/api/spec/shared_memory.h @@ -106,7 +106,7 @@ typedef struct odp_shm_capability_t { * * This number of separate shared memory blocks can be * reserved concurrently. */ - unsigned max_blocks; + unsigned int max_blocks; /** Maximum memory block size in bytes * diff --git a/include/odp/api/spec/version.h.in b/include/odp/api/spec/version.h.in index 35df9ba26..4c3cbfee8 100644 --- a/include/odp/api/spec/version.h.in +++ b/include/odp/api/spec/version.h.in @@ -4,7 +4,6 @@ * SPDX-License-Identifier: BSD-3-Clause */ - /** * @file * commit b904314ec95e4e680a739a5f98b6fd814b9a6fef Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Mon Oct 28 14:36:13 2019 +0200 example: cls: update help text Updated help text with changes to application options. Also, avoid shm memory leak when command line options have errors, or when --help option is used. Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Stanislaw Kardach <skardach(a)marvell.com> diff --git a/example/classifier/odp_classifier.c b/example/classifier/odp_classifier.c index ecfa732a9..3af418704 100644 --- a/example/classifier/odp_classifier.c +++ b/example/classifier/odp_classifier.c @@ -95,9 +95,9 @@ static appl_args_t *appl_args_gbl; static int drop_err_pkts(odp_packet_t pkt_tbl[], unsigned len); static void swap_pkt_addrs(odp_packet_t pkt_tbl[], unsigned len); -static void parse_args(int argc, char *argv[], appl_args_t *appl_args); +static int parse_args(int argc, char *argv[], appl_args_t *appl_args); static void print_info(char *progname, appl_args_t *appl_args); -static void usage(char *progname); +static void usage(void); static inline void print_cls_statistics(appl_args_t *args) { @@ -568,7 +568,8 @@ int main(int argc, char *argv[]) appl_args_gbl = args; memset(args, 0, sizeof(*args)); /* Parse and store the application arguments */ - parse_args(argc, argv, args); + if (parse_args(argc, argv, args)) + goto args_error; /* Print both system and application information */ print_info(NO_PATH(argv[0]), args); @@ -662,13 +663,14 @@ int main(int argc, char *argv[]) ODPH_ERR("err: odp_pool_destroy for %d\n", i); } - free(args->if_name); - odp_shm_free(shm); if (odp_pktio_close(pktio)) ODPH_ERR("err: close pktio error\n"); if (odp_pool_destroy(pool)) ODPH_ERR("err: odp_pool_destroy error\n"); +args_error: + odp_shm_free(shm); + ret = odp_term_local(); if (ret) ODPH_ERR("odp_term_local error %d\n", ret); @@ -772,7 +774,7 @@ static int convert_str_to_pmr_enum(char *token, odp_cls_pmr_term_t *term) return -1; } -static int parse_pmr_policy(appl_args_t *appl_args, char *argv[], char *optarg) +static int parse_pmr_policy(appl_args_t *appl_args, char *optarg) { int policy_count; char *token, *cos0, *cos1; @@ -889,7 +891,7 @@ static int parse_pmr_policy(appl_args_t *appl_args, char *argv[], char *optarg) return -1; break; default: - usage(argv[0]); + usage(); exit(EXIT_FAILURE); } @@ -923,7 +925,7 @@ static int parse_pmr_policy(appl_args_t *appl_args, char *argv[], char *optarg) * @param argv[] argument vector * @param appl_args Store application arguments here */ -static void parse_args(int argc, char *argv[], appl_args_t *appl_args) +static int parse_args(int argc, char *argv[], appl_args_t *appl_args) { int opt; int long_index; @@ -931,6 +933,7 @@ static void parse_args(int argc, char *argv[], appl_args_t *appl_args) int i; int interface = 0; int policy = 0; + int ret = 0; static const struct option longopts[] = { {"count", required_argument, NULL, 'c'}, @@ -960,7 +963,7 @@ static void parse_args(int argc, char *argv[], appl_args_t *appl_args) appl_args->cpu_count = atoi(optarg); break; case 'p': - if (0 > parse_pmr_policy(appl_args, argv, optarg)) + if (0 > parse_pmr_policy(appl_args, optarg)) continue; policy = 1; break; @@ -970,15 +973,15 @@ static void parse_args(int argc, char *argv[], appl_args_t *appl_args) case 'i': len = strlen(optarg); if (len == 0) { - usage(argv[0]); - exit(EXIT_FAILURE); + ret = -1; + break; } len += 1; /* add room for '\0' */ appl_args->if_name = malloc(len); if (appl_args->if_name == NULL) { - usage(argv[0]); - exit(EXIT_FAILURE); + ret = -1; + break; } strcpy(appl_args->if_name, optarg); @@ -995,24 +998,30 @@ static void parse_args(int argc, char *argv[], appl_args_t *appl_args) appl_args->verbose = 1; break; case 'h': - usage(argv[0]); - exit(EXIT_SUCCESS); + ret = -1; break; default: break; } } - if (!interface || !policy) { - usage(argv[0]); - exit(EXIT_FAILURE); - } - if (appl_args->if_name == NULL) { - usage(argv[0]); - exit(EXIT_FAILURE); + if (!interface || !policy) + ret = -1; + + if (appl_args->if_name == NULL) + ret = -1; + + if (ret) { + usage(); + + if (appl_args->if_name) + free(appl_args->if_name); } - optind = 1; /* reset 'extern optind' from the getopt lib */ + /* reset optind from the getopt lib */ + optind = 1; + + return ret; } /** @@ -1033,45 +1042,44 @@ static void print_info(char *progname, appl_args_t *appl_args) /** * Prinf usage information */ -static void usage(char *progname) +static void usage(void) { printf("\n" - "OpenDataPlane Classifier example.\n" - "Usage: %s OPTIONS\n" - " E.g. %s -i eth1 -m 0 -p \"ODP_PMR_SIP_ADDR:10.10.10.5:FFFFFFFF:queue1\" \\\n" - "\t\t\t-p \"ODP_PMR_SIP_ADDR:10.10.10.7:000000FF:queue2\" \\\n" - "\t\t\t-p \"ODP_PMR_SIP_ADDR:10.5.5.10:FFFFFF00:queue3\"\n" - "\n" - "For the above example configuration the following will be the packet distribution\n" - "queue1\t\tPackets with source ip address 10.10.10.5\n" - "queue2\t\tPackets with source ip address whose last 8 bits match 7\n" - "queue3\t\tPackets with source ip address in the subnet 10.5.5.0\n" - "\n" - "Mandatory OPTIONS:\n" - " -i, --interface Eth interface\n" - " -p, --policy [<odp_cls_pmr_term_t>|<offset>]:<value>:<mask bits>:<queue name>\n" - "\n" - "<odp_cls_pmr_term_t> Packet Matching Rule defined with odp_cls_pmr_term_t " - "for the policy\n" - "<offset> Absolute offset in bytes from frame start to define a " - "ODP_PMR_CUSTOM_FRAME Packet Matching Rule for the policy\n" - "\n" - "<value> PMR value to be matched.\n" - "\n" - "<mask bits> PMR mask bits to be applied on the PMR term value\n" - "\n" - "Optional OPTIONS\n" - " -c, --count <number> CPU count, 0=all available, default=1\n" - "\n" - " -m, --mode 0: Packet Drop mode. Received packets will be dropped\n" - " !0: Packet ICMP mode. Received packets will be sent back\n" - " default: Packet Drop mode\n" - "\n" - " -t, --timeout !0: Time for which the classifier will be run in seconds\n" - " 0: Runs in infinite loop\n" - " default: Runs in infinite loop\n" - "\n" - " -h, --help Display help and exit.\n" - "\n", NO_PATH(progname), NO_PATH(progname) - ); + "ODP Classifier example.\n" + "Usage: odp_classifier OPTIONS\n" + " E.g. odp_classifier -i eth1 -m 0 -p \"ODP_PMR_SIP_ADDR:10.10.10.0:0xFFFFFF00:queue1\" \\\n" + " -p \"ODP_PMR_SIP_ADDR:10.10.10.10:0xFFFFFFFF:queue1:queue2\"\n" + "\n" + "The above example would classify:\n" + " 1) Packets from source IP address 10.10.10.0/24 to queue1, except ...\n" + " 2) Packets from source IP address 10.10.10.10 to queue2\n" + " 3) All other packets to DefaultCos\n" + "\n" + "Mandatory OPTIONS:\n" + " -i, --interface <interface name>\n" + " -p, --policy <PMR term>:<offset>:<value>:<mask>:<src queue>:<dst queue>\n" + "\n" + " <PMR term> PMR term name defined in odp_cls_pmr_term_t\n" + " <offset> If term is ODP_PMR_CUSTOM_FRAME or _CUSTOM_L3, offset in bytes is used\n" + " <value> PMR value to be matched\n" + " <mask> PMR mask bits to be applied on the PMR value.\n" + " CUSTOM PMR terms accept plain hex string, other PMR terms require\n" + " hex string with '0x' prefix.\n" + " <src queue> Optional name of the source queue (CoS). The default CoS is used when\n" + " this is not defined.\n" + " <dst queue> Name of the destination queue (CoS).\n" + "\n" + "Optional OPTIONS\n" + " -c, --count <num> CPU count, 0=all available, default=1\n" + "\n" + " -m, --mode <mode> 0: Packet Drop mode. Received packets will be dropped\n" + " !0: Packet ICMP mode. Received packets will be sent back\n" + " default: Packet Drop mode\n" + "\n" + " -t, --timeout <sec> !0: Time for which the classifier will be run in seconds\n" + " 0: Runs in infinite loop\n" + " default: Runs in infinite loop\n" + "\n" + " -h, --help Display help and exit.\n" + "\n"); } commit 2376fa5b4b5f5b7b9d2c11cb08438f11883bfe50 Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Mon Oct 28 11:04:59 2019 +0200 example: cls: add ODP_PMR_CUSTOM_L3 support Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Stanislaw Kardach <skardach(a)marvell.com> diff --git a/example/classifier/odp_classifier.c b/example/classifier/odp_classifier.c index 47c689f6e..ecfa732a9 100644 --- a/example/classifier/odp_classifier.c +++ b/example/classifier/odp_classifier.c @@ -764,6 +764,9 @@ static int convert_str_to_pmr_enum(char *token, odp_cls_pmr_term_t *term) } else if (strcasecmp(token, "ODP_PMR_CUSTOM_FRAME") == 0) { *term = ODP_PMR_CUSTOM_FRAME; return 0; + } else if (strcasecmp(token, "ODP_PMR_CUSTOM_L3") == 0) { + *term = ODP_PMR_CUSTOM_L3; + return 0; } return -1; @@ -856,6 +859,8 @@ static int parse_pmr_policy(appl_args_t *appl_args, char *argv[], char *optarg) stats[policy_count].rule.val_sz = 4; break; case ODP_PMR_CUSTOM_FRAME: + /* Fall through */ + case ODP_PMR_CUSTOM_L3: /* :<offset>:<value>:<mask> */ token = strtok(NULL, ":"); errno = 0; commit 55ea0298a7cf917b466d2154d79d00fd86f3c936 Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Mon Oct 28 10:20:43 2019 +0200 linux-gen: cls: add ODP_PMR_CUSTOM_L3 support Added support for ODP_PMR_CUSTOM_L3 PMR term. Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Stanislaw Kardach <skardach(a)marvell.com> diff --git a/platform/linux-generic/odp_classification.c b/platform/linux-generic/odp_classification.c index 60171cc5e..1b5337ff6 100644 --- a/platform/linux-generic/odp_classification.c +++ b/platform/linux-generic/odp_classification.c @@ -174,6 +174,7 @@ int odp_cls_capability(odp_cls_capability_t *capability) capability->supported_terms.bit.sip6_addr = 1; capability->supported_terms.bit.dip6_addr = 1; capability->supported_terms.bit.custom_frame = 1; + capability->supported_terms.bit.custom_l3 = 1; capability->random_early_detection = ODP_SUPPORT_NO; capability->back_pressure = ODP_SUPPORT_NO; capability->threshold_red.all_bits = 0; @@ -631,6 +632,8 @@ static int pmr_create_term(pmr_term_value_t *value, break; case ODP_PMR_CUSTOM_FRAME: + /* Fall through */ + case ODP_PMR_CUSTOM_L3: custom = 1; size = MAX_PMR_TERM_SIZE; break; @@ -1050,7 +1053,8 @@ static inline int verify_pmr_custom_frame(const uint8_t *pkt_addr, odp_packet_hdr_t *pkt_hdr, pmr_term_value_t *term_value) { - uint64_t val = 0; + uint32_t i; + uint8_t val; uint32_t offset = term_value->offset; uint32_t val_sz = term_value->val_sz; @@ -1059,11 +1063,46 @@ static inline int verify_pmr_custom_frame(const uint8_t *pkt_addr, if (packet_len(pkt_hdr) <= offset + val_sz) return 0; - memcpy(&val, pkt_addr + offset, val_sz); - if (term_value->match.value == (val & term_value->match.mask)) - return 1; + pkt_addr += offset; - return 0; + for (i = 0; i < val_sz; i++) { + val = pkt_addr[i] & term_value->match.mask_u8[i]; + + if (val != term_value->match.value_u8[i]) + return 0; + } + + return 1; +} + +static inline int verify_pmr_custom_l3(const uint8_t *pkt_addr, + odp_packet_hdr_t *pkt_hdr, + pmr_term_value_t *term_value) +{ + uint32_t i; + uint8_t val; + uint32_t l3_offset = pkt_hdr->p.l3_offset; + uint32_t offset = l3_offset + term_value->offset; + uint32_t val_sz = term_value->val_sz; + + ODP_ASSERT(val_sz <= MAX_PMR_TERM_SIZE); + + if (!pkt_hdr->p.input_flags.l3) + return 0; + + if (packet_len(pkt_hdr) <= offset + val_sz) + return 0; + + pkt_addr += offset; + + for (i = 0; i < val_sz; i++) { + val = pkt_addr[i] & term_value->match.mask_u8[i]; + + if (val != term_value->match.value_u8[i]) + return 0; + } + + return 1; } static inline int verify_pmr_eth_type_0(const uint8_t *pkt_addr, @@ -1222,6 +1261,11 @@ static int verify_pmr(pmr_t *pmr, const uint8_t *pkt_addr, term_value)) pmr_failure = 1; break; + case ODP_PMR_CUSTOM_L3: + if (!verify_pmr_custom_l3(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; case ODP_PMR_INNER_HDR_OFF: break; default: commit d688a269562ff9efc2ff03411c385a2bd1228d2c Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Mon Oct 14 14:13:33 2019 +0300 linux-gen: cls: add custom frame PMR capability Mark custom frame as a supported PMR. Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Stanislaw Kardach <skardach(a)marvell.com> diff --git a/platform/linux-generic/odp_classification.c b/platform/linux-generic/odp_classification.c index 164daf3fc..60171cc5e 100644 --- a/platform/linux-generic/odp_classification.c +++ b/platform/linux-generic/odp_classification.c @@ -173,6 +173,7 @@ int odp_cls_capability(odp_cls_capability_t *capability) capability->supported_terms.bit.dip_addr = 1; capability->supported_terms.bit.sip6_addr = 1; capability->supported_terms.bit.dip6_addr = 1; + capability->supported_terms.bit.custom_frame = 1; capability->random_early_detection = ODP_SUPPORT_NO; capability->back_pressure = ODP_SUPPORT_NO; capability->threshold_red.all_bits = 0; commit 13b478eaa76ee8461341b114d447835e3906748d Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Mon Oct 14 14:12:42 2019 +0300 validation: cls: add custom PMR test cases Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Stanislaw Kardach <skardach(a)marvell.com> diff --git a/test/validation/api/classification/odp_classification_test_pmr.c b/test/validation/api/classification/odp_classification_test_pmr.c index 46bbdf4b1..d4a3110e3 100644 --- a/test/validation/api/classification/odp_classification_test_pmr.c +++ b/test/validation/api/classification/odp_classification_test_pmr.c @@ -2037,6 +2037,147 @@ static void classification_test_pmr_term_tcp_dport_multi(void) _classification_test_pmr_term_tcp_dport(SHM_PKT_NUM_BUFS / 4); } +static void test_pmr_term_custom(int custom_l3) +{ + odp_packet_t pkt; + uint32_t seqno; + int retval; + odp_pktio_t pktio; + odp_queue_t queue; + odp_queue_t retqueue; + odp_queue_t default_queue; + odp_pool_t pool; + odp_pool_t default_pool; + odp_pmr_t pmr; + odp_cos_t cos; + odp_cos_t default_cos; + uint32_t dst_addr, src_addr; + uint32_t addr_be, mask_be; + uint32_t dst_mask, src_mask; + char cosname[ODP_QUEUE_NAME_LEN]; + odp_pmr_param_t pmr_param; + odp_cls_cos_param_t cls_param; + odph_ipv4hdr_t *ip; + odph_ethhdr_t *eth; + const char *pmr_src_str = "10.0.8.0/24"; + const char *pmr_dst_str = "10.0.9.0/24"; + const char *pkt_src_str = "10.0.8.88/32"; + const char *pkt_dst_str = "10.0.9.99/32"; + + pktio = create_pktio(ODP_QUEUE_TYPE_SCHED, pkt_pool, true); + retval = start_pktio(pktio); + CU_ASSERT(retval == 0); + + configure_default_cos(pktio, &default_cos, + &default_queue, &default_pool); + + queue = queue_create("ipv4 addr", true); + CU_ASSERT_FATAL(queue != ODP_QUEUE_INVALID); + + pool = pool_create("ipv4 addr"); + CU_ASSERT_FATAL(pool != ODP_POOL_INVALID); + + sprintf(cosname, "ipv4 addr"); + odp_cls_cos_param_init(&cls_param); + cls_param.pool = pool; + cls_param.queue = queue; + cls_param.drop_policy = ODP_COS_DROP_POOL; + + cos = odp_cls_cos_create(cosname, &cls_param); + CU_ASSERT_FATAL(cos != ODP_COS_INVALID); + + /* Match values for custom PRM rules are passed in network endian */ + parse_ipv4_string(pmr_src_str, &src_addr, &src_mask); + parse_ipv4_string(pmr_dst_str, &dst_addr, &dst_mask); + + odp_cls_pmr_param_init(&pmr_param); + + if (custom_l3) { + addr_be = odp_cpu_to_be_32(dst_addr); + mask_be = odp_cpu_to_be_32(dst_mask); + pmr_param.term = ODP_PMR_CUSTOM_L3; + pmr_param.match.value = &addr_be; + pmr_param.match.mask = &mask_be; + pmr_param.val_sz = sizeof(addr_be); + /* Offset from start of L3 to IPv4 dst address */ + pmr_param.offset = 16; + } else { + addr_be = odp_cpu_to_be_32(src_addr); + mask_be = odp_cpu_to_be_32(src_mask); + pmr_param.term = ODP_PMR_CUSTOM_FRAME; + pmr_param.match.value = &addr_be; + pmr_param.match.mask = &mask_be; + pmr_param.val_sz = sizeof(addr_be); + /* Offset from start of ethernet/IPv4 frame to IPv4 + * src address */ + pmr_param.offset = 26; + } + + pmr = odp_cls_pmr_create(&pmr_param, 1, default_cos, cos); + CU_ASSERT_FATAL(pmr != ODP_PMR_INVALID); + + /* IPv4 packet with matching addresses */ + parse_ipv4_string(pkt_src_str, &src_addr, NULL); + parse_ipv4_string(pkt_dst_str, &dst_addr, NULL); + pkt = create_packet(default_pkt_info); + CU_ASSERT_FATAL(pkt != ODP_PACKET_INVALID); + eth = (odph_ethhdr_t *)odp_packet_l2_ptr(pkt, NULL); + odp_pktio_mac_addr(pktio, eth->src.addr, ODPH_ETHADDR_LEN); + odp_pktio_mac_addr(pktio, eth->dst.addr, ODPH_ETHADDR_LEN); + ip = (odph_ipv4hdr_t *)odp_packet_l3_ptr(pkt, NULL); + ip->src_addr = odp_cpu_to_be_32(src_addr); + ip->dst_addr = odp_cpu_to_be_32(dst_addr); + odph_ipv4_csum_update(pkt); + + seqno = cls_pkt_get_seq(pkt); + CU_ASSERT(seqno != TEST_SEQ_INVALID); + + enqueue_pktio_interface(pkt, pktio); + + pkt = receive_packet(&retqueue, ODP_TIME_SEC_IN_NS); + CU_ASSERT_FATAL(pkt != ODP_PACKET_INVALID); + CU_ASSERT(seqno == cls_pkt_get_seq(pkt)); + CU_ASSERT(retqueue == queue); + odp_packet_free(pkt); + + /* Other packets delivered to default queue */ + pkt = create_packet(default_pkt_info); + CU_ASSERT_FATAL(pkt != ODP_PACKET_INVALID); + seqno = cls_pkt_get_seq(pkt); + CU_ASSERT(seqno != TEST_SEQ_INVALID); + eth = (odph_ethhdr_t *)odp_packet_l2_ptr(pkt, NULL); + odp_pktio_mac_addr(pktio, eth->src.addr, ODPH_ETHADDR_LEN); + odp_pktio_mac_addr(pktio, eth->dst.addr, ODPH_ETHADDR_LEN); + + enqueue_pktio_interface(pkt, pktio); + + pkt = receive_packet(&retqueue, ODP_TIME_SEC_IN_NS); + CU_ASSERT_FATAL(pkt != ODP_PACKET_INVALID); + CU_ASSERT(seqno == cls_pkt_get_seq(pkt)); + CU_ASSERT(retqueue == default_queue); + + odp_cos_destroy(cos); + odp_cos_destroy(default_cos); + odp_cls_pmr_destroy(pmr); + odp_packet_free(pkt); + stop_pktio(pktio); + odp_pool_destroy(default_pool); + odp_pool_destroy(pool); + odp_queue_destroy(queue); + odp_queue_destroy(default_queue); + odp_pktio_close(pktio); +} + +static void classification_test_pmr_term_custom_frame(void) +{ + test_pmr_term_custom(0); +} + +static void classification_test_pmr_term_custom_l3(void) +{ + test_pmr_term_custom(1); +} + static int check_capa_tcp_dport(void) { return cls_capa.supported_terms.bit.tcp_dport; @@ -2112,6 +2253,16 @@ static int check_capa_ethtype_x(void) return cls_capa.supported_terms.bit.ethtype_x; } +static int check_capa_custom_frame(void) +{ + return cls_capa.supported_terms.bit.custom_frame; +} + +static int check_capa_custom_l3(void) +{ + return cls_capa.supported_terms.bit.custom_l3; +} + odp_testinfo_t classification_suite_pmr[] = { ODP_TEST_INFO_CONDITIONAL(classification_test_pmr_term_tcp_dport, check_capa_tcp_dport), @@ -2145,6 +2296,10 @@ odp_testinfo_t classification_suite_pmr[] = { check_capa_ethtype_0), ODP_TEST_INFO_CONDITIONAL(classification_test_pmr_term_eth_type_x, check_capa_ethtype_x), + ODP_TEST_INFO_CONDITIONAL(classification_test_pmr_term_custom_frame, + check_capa_custom_frame), + ODP_TEST_INFO_CONDITIONAL(classification_test_pmr_term_custom_l3, + check_capa_custom_l3), ODP_TEST_INFO(classification_test_pktin_classifier_flag), ODP_TEST_INFO(classification_test_pmr_term_tcp_dport_multi), ODP_TEST_INFO_NULL, commit 4c4145f8c9c73665c9f834f8ca639b9f0525e30f Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Fri Oct 25 15:07:25 2019 +0300 api: cls: add ODP_PMR_CUSTOM_L3 term Added new term to match custom layer 3 protocol fields. PMR offset refers to the start of layer 3 in the packet. Other PMR rules (e.g. L2 classification rules) may precede custom L3 rules. Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Stanislaw Kardach <skardach(a)marvell.com> diff --git a/include/odp/api/spec/classification.h b/include/odp/api/spec/classification.h index 67fdd20aa..eadcaa2d8 100644 --- a/include/odp/api/spec/classification.h +++ b/include/odp/api/spec/classification.h @@ -97,11 +97,12 @@ typedef union odp_cls_pmr_terms_t { uint64_t ipsec_spi:1; /** NVGRE/VXLAN network identifier */ uint64_t ld_vni:1; - /** Custom match rule, offset from start of - * frame. The match is defined by the offset, the - * expected value, and its size. - */ - uint64_t custom_frame:1; + /** Custom frame match rule. PMR offset is counted from + * the start of the packet. */ + uint64_t custom_frame:1; + /** Custom layer 3 match rule. PMR offset is counted from + * the start of layer 3 in the packet. */ + uint64_t custom_l3:1; } bit; /** All bits of the bit field structure */ @@ -535,6 +536,15 @@ typedef enum { */ ODP_PMR_CUSTOM_FRAME, + /** + * Custom layer 3 match rule + * + * PMR offset is counted from the start of layer 3 in the packet. + * The match is defined by the offset, the expected value, and its size. + * Custom L3 rules may be combined with other PMRs. + */ + ODP_PMR_CUSTOM_L3, + /** Inner header may repeat above values with this offset */ ODP_PMR_INNER_HDR_OFF = 32 @@ -578,11 +588,18 @@ typedef struct odp_pmr_param_t { const void *val_end; } range; }; - uint32_t val_sz; /**< Size of the term value */ - uint32_t offset; /**< User-defined offset in packet - Used if term == ODP_PMR_CUSTOM_FRAME only, - ignored otherwise */ + /** Size of the value to be matched */ + uint32_t val_sz; + + /** Offset to the value + * + * Byte offset to the value to be matched in a packet. PMR term defines + * starting point for the offset. Used only with custom PMR terms, + * ignored with other terms. + */ + uint32_t offset; + } odp_pmr_param_t; /** commit 4c58ce0cd530c59907164f2fa7ae78800c494151 Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Fri Oct 25 14:36:08 2019 +0300 example: cls: change PMR values and masks to big endian Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Stanislaw Kardach <skardach(a)marvell.com> diff --git a/example/classifier/odp_classifier.c b/example/classifier/odp_classifier.c index 691fec297..47c689f6e 100644 --- a/example/classifier/odp_classifier.c +++ b/example/classifier/odp_classifier.c @@ -44,6 +44,9 @@ */ #define DISPLAY_STRING_LEN 32 +/** Maximum PMR value size */ +#define MAX_VAL_SIZE 16 + /** Get rid of path in filename - only for unix-type paths using '/' */ #define NO_PATH(file_name) (strrchr((file_name), '/') ? \ strrchr((file_name), '/') + 1 : (file_name)) @@ -59,8 +62,8 @@ typedef struct { char src_cos_name[ODP_COS_NAME_LEN]; /**< source cos name */ struct { odp_cls_pmr_term_t term; /**< odp pmr term value */ - uint64_t val; /**< pmr term value */ - uint64_t mask; /**< pmr term mask */ + uint8_t value_be[MAX_VAL_SIZE]; /**< value in big endian */ + uint8_t mask_be[MAX_VAL_SIZE]; /**< mask in big endian */ uint32_t val_sz; /**< size of the pmr term */ uint32_t offset; /**< pmr term offset */ } rule; @@ -166,39 +169,20 @@ static inline void print_cls_statistics(appl_args_t *args) printf("\n"); } -static inline int parse_mask(const char *str, uint64_t *mask) -{ - uint64_t b; - int ret; - - ret = sscanf(str, "%" SCNx64, &b); - *mask = b; - return ret != 1; -} - -static int parse_value(const char *str, uint64_t *val, uint32_t *val_sz) +static int parse_custom(const char *str, uint8_t *buf_be, int max_size) { - size_t len; - size_t i; - int converted; - union { - uint64_t u64; - uint8_t u8[8]; - } buf = {.u64 = 0}; + int i, len; + /* hex string without 0x prefix */ len = strlen(str); - if (len > 2 * sizeof(buf)) + if (len > 2 * max_size) return -1; - for (i = 0; i < len; i += 2) { - converted = sscanf(&str[i], "%2" SCNx8, &buf.u8[i / 2]); - if (1 != converted) + for (i = 0; i < len; i += 2) + if (sscanf(&str[i], "%2" SCNx8, &buf_be[i / 2]) != 1) return -1; - } - *val = buf.u64; - *val_sz = len / 2; - return 0; + return len / 2; } /** @@ -494,8 +478,8 @@ static void configure_cos(odp_cos_t default_cos, appl_args_t *args) odp_cls_pmr_param_init(&pmr_param); pmr_param.term = stats->rule.term; - pmr_param.match.value = &stats->rule.val; - pmr_param.match.mask = &stats->rule.mask; + pmr_param.match.value = stats->rule.value_be; + pmr_param.match.mask = stats->rule.mask_be; pmr_param.val_sz = stats->rule.val_sz; pmr_param.offset = stats->rule.offset; @@ -793,9 +777,10 @@ static int parse_pmr_policy(appl_args_t *appl_args, char *argv[], char *optarg) odp_cls_pmr_term_t term; global_statistics *stats; char *pmr_str; - uint32_t offset; - uint32_t ip_addr; - unsigned long int value; + uint32_t offset, ip_addr, u32; + unsigned long int value, mask; + uint16_t u16; + int val_sz, mask_sz; policy_count = appl_args->policy_count; stats = appl_args->stats; @@ -832,12 +817,18 @@ static int parse_pmr_policy(appl_args_t *appl_args, char *argv[], char *optarg) strncpy(stats[policy_count].value, token, DISPLAY_STRING_LEN - 1); value = strtoul(token, NULL, 0); - stats[policy_count].rule.val = value; + u16 = value; + u16 = odp_cpu_to_be_16(u16); + memcpy(stats[policy_count].rule.value_be, &u16, sizeof(u16)); token = strtok(NULL, ":"); strncpy(stats[policy_count].mask, token, DISPLAY_STRING_LEN - 1); - parse_mask(token, &stats[policy_count].rule.mask); + mask = strtoul(token, NULL, 0); + u16 = mask; + u16 = odp_cpu_to_be_16(u16); + memcpy(stats[policy_count].rule.mask_be, &u16, sizeof(u16)); + stats[policy_count].rule.val_sz = 2; break; case ODP_PMR_SIP_ADDR: @@ -851,12 +842,17 @@ static int parse_pmr_policy(appl_args_t *appl_args, char *argv[], char *optarg) exit(EXIT_FAILURE); } - stats[policy_count].rule.val = ip_addr; + u32 = odp_cpu_to_be_32(ip_addr); + memcpy(stats[policy_count].rule.value_be, &u32, sizeof(u32)); token = strtok(NULL, ":"); strncpy(stats[policy_count].mask, token, DISPLAY_STRING_LEN - 1); - parse_mask(token, &stats[policy_count].rule.mask); + mask = strtoul(token, NULL, 0); + u32 = mask; + u32 = odp_cpu_to_be_32(u32); + memcpy(stats[policy_count].rule.mask_be, &u32, sizeof(u32)); + stats[policy_count].rule.val_sz = 4; break; case ODP_PMR_CUSTOM_FRAME: @@ -864,19 +860,28 @@ static int parse_pmr_policy(appl_args_t *appl_args, char *argv[], char *optarg) token = strtok(NULL, ":"); errno = 0; offset = strtoul(token, NULL, 0); + stats[policy_count].rule.offset = offset; if (errno) return -1; token = strtok(NULL, ":"); strncpy(stats[policy_count].value, token, DISPLAY_STRING_LEN - 1); - parse_value(token, &stats[policy_count].rule.val, - &stats[policy_count].rule.val_sz); + val_sz = parse_custom(token, + stats[policy_count].rule.value_be, + MAX_VAL_SIZE); + stats[policy_count].rule.val_sz = val_sz; + if (val_sz <= 0) + return -1; + token = strtok(NULL, ":"); strncpy(stats[policy_count].mask, token, DISPLAY_STRING_LEN - 1); - parse_mask(token, &stats[policy_count].rule.mask); - stats[policy_count].rule.offset = offset; + mask_sz = parse_custom(token, + stats[policy_count].rule.mask_be, + MAX_VAL_SIZE); + if (mask_sz != val_sz) + return -1; break; default: usage(argv[0]); commit 65891255eec4884e941aece9808ad684155feac8 Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Fri Oct 25 11:08:03 2019 +0300 linux-gen: cls: clean up IPv6 match implementation Use the same range data structure for all terms (removed IPv6 specific fields). Range terms are not currently supported. Return failure on PMR create if range is used. Removed other range related code as has not been tested and is not needed currently. Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Stanislaw Kardach <skardach(a)marvell.com> diff --git a/platform/linux-generic/include/odp_classification_datamodel.h b/platform/linux-generic/include/odp_classification_datamodel.h index 1032a1c0b..9ebb2a509 100644 --- a/platform/linux-generic/include/odp_classification_datamodel.h +++ b/platform/linux-generic/include/odp_classification_datamodel.h @@ -42,8 +42,8 @@ extern "C" { #define CLS_COS_L3_QOS_BITS 6 /* Max L3 QoS Value */ #define CLS_COS_MAX_L3_QOS (1 << CLS_COS_L3_QOS_BITS) -/* Max PMR Term bits */ -#define CLS_PMR_TERM_BYTES_MAX 16 +/* Max PMR Term size */ +#define MAX_PMR_TERM_SIZE 16 /* Max queue per Class of service */ #define CLS_COS_QUEUE_MAX 32 /* Max number of implementation created queues */ @@ -61,39 +61,62 @@ typedef union { }; } odp_cls_hash_proto_t; -/** -Packet Matching Rule Term Value - -Stores the Term and Value mapping for a PMR. -The maximum size of value currently supported in 64 bits -**/ +/* + * Term and value mapping for a PMR + */ typedef struct pmr_term_value { - odp_cls_pmr_term_t term; /* PMR Term */ - odp_bool_t range_term; /* True if range, false if match */ + /* PMR Term */ + odp_cls_pmr_term_t term; + + /* True if range, false if match */ + odp_bool_t range_term; + union { + /* Match value and mask */ struct { - /** Value to be matched */ - uint64_t value; - /** Masked set of bits to be matched */ - uint64_t mask; + /* Value to be matched. Arrays are used with custom and + * IPv6 address terms. */ + union { + uint64_t value; + uint8_t value_u8[MAX_PMR_TERM_SIZE]; + uint64_t value_u64[2]; + }; + + /* Mask for the data to be matched */ + union { + uint64_t mask; + uint8_t mask_u8[MAX_PMR_TERM_SIZE]; + uint64_t mask_u64[2]; + }; + } match; + + /* Range values */ struct { - /** Start value of the range */ - uint64_t val_start; - /** End value of the range */ - uint64_t val_end; + /* Start value of the range */ + union { + uint64_t start; + uint8_t start_u8[MAX_PMR_TERM_SIZE]; + uint64_t start_u64[2]; + }; + + /* End value of the range */ + union { + uint64_t end; + uint8_t end_u8[MAX_PMR_TERM_SIZE]; + uint64_t end_u64[2]; + }; + } range; - struct { - _odp_ipv6_addr_t addr; - _odp_ipv6_addr_t mask; - } match_ipv6; - struct { - _odp_ipv6_addr_t addr_start; - _odp_ipv6_addr_t addr_end; - } range_ipv6; + }; - uint32_t offset; /**< Offset if term == ODP_PMR_CUSTOM_FRAME */ - uint32_t val_sz; /**< Size of the value to be matched */ + + /* Offset used with custom PMR */ + uint32_t offset; + + /* Size of the value to be matched */ + uint32_t val_sz; + } pmr_term_value_t; /* diff --git a/platform/linux-generic/odp_classification.c b/platform/linux-generic/odp_classification.c index e85e8d00c..164daf3fc 100644 --- a/platform/linux-generic/odp_classification.c +++ b/platform/linux-generic/odp_classification.c @@ -576,6 +576,11 @@ static int pmr_create_term(pmr_term_value_t *value, int custom = 0; odp_cls_pmr_term_t term = param->term; + if (param->range_term) { + ODP_ERR("PRM value range not supported\n"); + return -1; + } + value->term = term; value->range_term = param->range_term; @@ -626,7 +631,7 @@ static int pmr_create_term(pmr_term_value_t *value, case ODP_PMR_CUSTOM_FRAME: custom = 1; - size = CLS_PMR_TERM_BYTES_MAX; + size = MAX_PMR_TERM_SIZE; break; default: @@ -640,45 +645,14 @@ static int pmr_create_term(pmr_term_value_t *value, return -1; } - switch (value->term) { - case ODP_PMR_SIP6_ADDR: - case ODP_PMR_DIP6_ADDR: - if (!value->range_term) { - memset(value->match_ipv6.addr.u8, 0, 16); - memset(value->match_ipv6.mask.u8, 0, 16); - memcpy(&value->match_ipv6.addr.u8, param->match.value, - param->val_sz); - memcpy(&value->match_ipv6.mask.u8, param->match.mask, - param->val_sz); - for (i = 0; i < 2; i++) - value->match_ipv6.addr.u64[i] &= - value->match_ipv6.mask.u64[i]; - } else { - memset(value->range_ipv6.addr_start.u8, 0, 16); - memset(value->range_ipv6.addr_end.u8, 0, 16); - memcpy(&value->range_ipv6.addr_start.u8, param->range.val_start, - param->val_sz); - memcpy(&value->range_ipv6.addr_end.u8, param->range.val_end, - param->val_sz); - } + memset(&value->match.value, 0, MAX_PMR_TERM_SIZE); + memset(&value->match.mask, 0, MAX_PMR_TERM_SIZE); + memcpy(&value->match.value, param->match.value, param->val_sz); + memcpy(&value->match.mask, param->match.mask, param->val_sz); + + for (i = 0; i < param->val_sz; i++) + value->match.value_u8[i] &= value->match.mask_u8[i]; - break; - default: - if (!value->range_term) { - value->match.value = 0; - value->match.mask = 0; - memcpy(&value->match.value, param->match.value, param->val_sz); - memcpy(&value->match.mask, param->match.mask, param->val_sz); - value->match.value &= value->match.mask; - } else { - value->range.val_start = 0; - value->range.val_end = 0; - memcpy(&value->range.val_start, param->range.val_start, - param->val_sz); - memcpy(&value->range.val_end, param->range.val_end, - param->val_sz); - } - } value->offset = param->offset; value->val_sz = param->val_sz; return 0; @@ -952,16 +926,13 @@ static inline int verify_pmr_ipv6_saddr(const uint8_t *pkt_addr, return 0; ipv6 = (const _odp_ipv6hdr_t *)(pkt_addr + pkt_hdr->p.l3_offset); + memcpy(addr, ipv6->src_addr.u64, _ODP_IPV6ADDR_LEN); - addr[0] = ipv6->src_addr.u64[0]; - addr[1] = ipv6->src_addr.u64[1]; - - /* 128 bit address is processed as two 64 bit value - * for bitwise AND operation */ - addr[0] = addr[0] & term_value->match_ipv6.mask.u64[0]; - addr[1] = addr[1] & term_value->match_ipv6.mask.u64[1]; + addr[0] = addr[0] & term_value->match.mask_u64[0]; + addr[1] = addr[1] & term_value->match.mask_u64[1]; - if (!memcmp(addr, term_value->match_ipv6.addr.u8, _ODP_IPV6ADDR_LEN)) + if (addr[0] == term_value->match.value_u64[0] && + addr[1] == term_value->match.value_u64[1]) return 1; return 0; @@ -976,16 +947,15 @@ static inline int verify_pmr_ipv6_daddr(const uint8_t *pkt_addr, if (!packet_hdr_has_ipv6(pkt_hdr)) return 0; + ipv6 = (const _odp_ipv6hdr_t *)(pkt_addr + pkt_hdr->p.l3_offset); - addr[0] = ipv6->dst_addr.u64[0]; - addr[1] = ipv6->dst_addr.u64[1]; + memcpy(addr, ipv6->dst_addr.u64, _ODP_IPV6ADDR_LEN); - /* 128 bit address is processed as two 64 bit value - * for bitwise AND operation */ - addr[0] = addr[0] & term_value->match_ipv6.mask.u64[0]; - addr[1] = addr[1] & term_value->match_ipv6.mask.u64[1]; + addr[0] = addr[0] & term_value->match.mask_u64[0]; + addr[1] = addr[1] & term_value->match.mask_u64[1]; - if (!memcmp(addr, term_value->match_ipv6.addr.u8, _ODP_IPV6ADDR_LEN)) + if (addr[0] == term_value->match.value_u64[0] && + addr[1] == term_value->match.value_u64[1]) return 1; return 0; @@ -1083,7 +1053,7 @@ static inline int verify_pmr_custom_frame(const uint8_t *pkt_addr, uint32_t offset = term_value->offset; uint32_t val_sz = term_value->val_sz; - ODP_ASSERT(val_sz <= CLS_PMR_TERM_BYTES_MAX); + ODP_ASSERT(val_sz <= MAX_PMR_TERM_SIZE); if (packet_len(pkt_hdr) <= offset + val_sz) return 0; commit d4eb0481e1dd40a0d0b70486cdf8f9b676862b2f Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Thu Oct 24 16:16:12 2019 +0300 linux-gen: cls: change ODP_PMR_ IPv4 addresses to big endian Change ODP_PMR_SIP_ADDR and ODP_PMR_DIP_ADDR to big endian. Both are changed simultaneously as some tests use both terms. Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Stanislaw Kardach <skardach(a)marvell.com> diff --git a/platform/linux-generic/odp_classification.c b/platform/linux-generic/odp_classification.c index ae90ee82a..e85e8d00c 100644 --- a/platform/linux-generic/odp_classification.c +++ b/platform/linux-generic/odp_classification.c @@ -825,7 +825,7 @@ static inline int verify_pmr_ipv4_saddr(const uint8_t *pkt_addr, if (!pkt_hdr->p.input_flags.ipv4) return 0; ip = (const _odp_ipv4hdr_t *)(pkt_addr + pkt_hdr->p.l3_offset); - ipaddr = odp_be_to_cpu_32(ip->src_addr); + ipaddr = ip->src_addr; if (term_value->match.value == (ipaddr & term_value->match.mask)) return 1; @@ -842,7 +842,7 @@ static inline int verify_pmr_ipv4_daddr(const uint8_t *pkt_addr, if (!pkt_hdr->p.input_flags.ipv4) return 0; ip = (const _odp_ipv4hdr_t *)(pkt_addr + pkt_hdr->p.l3_offset); - ipaddr = odp_be_to_cpu_32(ip->dst_addr); + ipaddr = ip->dst_addr; if (term_value->match.value == (ipaddr & term_value->match.mask)) return 1; diff --git a/test/validation/api/classification/odp_classification_test_pmr.c b/test/validation/api/classification/odp_classification_test_pmr.c index a939bdbee..46bbdf4b1 100644 --- a/test/validation/api/classification/odp_classification_test_pmr.c +++ b/test/validation/api/classification/odp_classification_test_pmr.c @@ -1722,6 +1722,10 @@ static void test_pmr_term_ipv4_addr(int dst) parse_ipv4_string(src_str, &src_addr, &src_mask); parse_ipv4_string(dst_str, &dst_addr, &dst_mask); + src_addr = odp_cpu_to_be_32(src_addr); + src_mask = odp_cpu_to_be_32(src_mask); + dst_addr = odp_cpu_to_be_32(dst_addr); + dst_mask = odp_cpu_to_be_32(dst_mask); odp_cls_pmr_param_init(&pmr_param); @@ -1748,8 +1752,8 @@ static void test_pmr_term_ipv4_addr(int dst) odp_pktio_mac_addr(pktio, eth->src.addr, ODPH_ETHADDR_LEN); odp_pktio_mac_addr(pktio, eth->dst.addr, ODPH_ETHADDR_LEN); ip = (odph_ipv4hdr_t *)odp_packet_l3_ptr(pkt, NULL); - ip->src_addr = odp_cpu_to_be_32(src_addr); - ip->dst_addr = odp_cpu_to_be_32(dst_addr); + ip->src_addr = src_addr; + ip->dst_addr = dst_addr; odph_ipv4_csum_update(pkt); seqno = cls_pkt_get_seq(pkt); diff --git a/test/validation/api/classification/odp_classification_tests.c b/test/validation/api/classification/odp_classification_tests.c index 3a97023cf..b86bfe272 100644 --- a/test/validation/api/classification/odp_classification_tests.c +++ b/test/validation/api/classification/odp_classification_tests.c @@ -213,6 +213,9 @@ void configure_cls_pmr_chain(void) CU_ASSERT_FATAL(cos_list[CLS_PMR_CHAIN_DST] != ODP_COS_INVALID); parse_ipv4_string(CLS_PMR_CHAIN_SADDR, &addr, &mask); + addr = odp_cpu_to_be_32(addr); + mask = odp_cpu_to_be_32(mask); + odp_cls_pmr_param_init(&pmr_param); pmr_param.term = ODP_PMR_SIP_ADDR; pmr_param.match.value = &addr; @@ -649,6 +652,9 @@ void configure_pktio_pmr_composite(void) CU_ASSERT_FATAL(cos_list[CLS_PMR_SET] != ODP_COS_INVALID); parse_ipv4_string(CLS_PMR_SET_SADDR, &addr, &mask); + addr = odp_cpu_to_be_32(addr); + mask = odp_cpu_to_be_32(mask); + odp_cls_pmr_param_init(&pmr_params[0]); pmr_params[0].term = ODP_PMR_SIP_ADDR; pmr_params[0].match.value = &addr; commit f15da1c0ee13dd0e342944dccdd9014c16d4d5be Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Thu Oct 24 15:22:38 2019 +0300 linux-gen: cls: change ODP_PMR_UDP/TCP ports to big endian Change ODP_PMR_UDP_DPORT, ODP_PMR_TCP_DPORT, ODP_PMR_UDP_SPORT and ODP_PMR_TCP_SPORT terms to big endian. All four needs to be changed simultaneously in order not to break some validation tests. Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Stanislaw Kardach <skardach(a)marvell.com> diff --git a/platform/linux-generic/odp_classification.c b/platform/linux-generic/odp_classification.c index 9ad3380e5..ae90ee82a 100644 --- a/platform/linux-generic/odp_classification.c +++ b/platform/linux-generic/odp_classification.c @@ -859,7 +859,7 @@ static inline int verify_pmr_tcp_sport(const uint8_t *pkt_addr, if (!pkt_hdr->p.input_flags.tcp) return 0; tcp = (const _odp_tcphdr_t *)(pkt_addr + pkt_hdr->p.l4_offset); - sport = odp_be_to_cpu_16(tcp->src_port); + sport = tcp->src_port; if (term_value->match.value == (sport & term_value->match.mask)) return 1; @@ -876,7 +876,7 @@ static inline int verify_pmr_tcp_dport(const uint8_t *pkt_addr, if (!pkt_hdr->p.input_flags.tcp) return 0; tcp = (const _odp_tcphdr_t *)(pkt_addr + pkt_hdr->p.l4_offset); - dport = odp_be_to_cpu_16(tcp->dst_port); + dport = tcp->dst_port; if (term_value->match.value == (dport & term_value->match.mask)) return 1; @@ -893,7 +893,7 @@ static inline int verify_pmr_udp_dport(const uint8_t *pkt_addr, if (!pkt_hdr->p.input_flags.udp) return 0; udp = (const _odp_udphdr_t *)(pkt_addr + pkt_hdr->p.l4_offset); - dport = odp_be_to_cpu_16(udp->dst_port); + dport = udp->dst_port; if (term_value->match.value == (dport & term_value->match.mask)) return 1; @@ -910,7 +910,7 @@ static inline int verify_pmr_udp_sport(const uint8_t *pkt_addr, if (!pkt_hdr->p.input_flags.udp) return 0; udp = (const _odp_udphdr_t *)(pkt_addr + pkt_hdr->p.l4_offset); - sport = odp_be_to_cpu_16(udp->src_port); + sport = udp->src_port; if (term_value->match.value == (sport & term_value->match.mask)) return 1; diff --git a/test/validation/api/classification/odp_classification_test_pmr.c b/test/validation/api/classification/odp_classification_test_pmr.c index 9ac735020..a939bdbee 100644 --- a/test/validation/api/classification/odp_classification_test_pmr.c +++ b/test/validation/api/classification/odp_classification_test_pmr.c @@ -121,8 +121,8 @@ static void classification_test_pktin_classifier_flag(void) odp_pmr_param_t pmr_param; odph_ethhdr_t *eth; - val = CLS_DEFAULT_DPORT; - mask = 0xffff; + val = odp_cpu_to_be_16(CLS_DEFAULT_DPORT); + mask = odp_cpu_to_be_16(0xffff); seqno = 0; /* classifier is disabled in pktin queue configuration */ @@ -167,7 +167,7 @@ static void classification_test_pktin_classifier_flag(void) odp_pktio_mac_addr(pktio, eth->dst.addr, ODPH_ETHADDR_LEN); tcp = (odph_tcphdr_t *)odp_packet_l4_ptr(pkt, NULL); - tcp->dst_port = odp_cpu_to_be_16(CLS_DEFAULT_DPORT); + tcp->dst_port = val; enqueue_pktio_interface(pkt, pktio); @@ -217,8 +217,8 @@ static void _classification_test_pmr_term_tcp_dport(int num_pkt) odp_pool_t pool_recv; odp_pmr_param_t pmr_param; odph_ethhdr_t *eth; - val = CLS_DEFAULT_DPORT; - mask = 0xffff; + val = odp_cpu_to_be_16(CLS_DEFAULT_DPORT); + mask = odp_cpu_to_be_16(0xffff); pktio = create_pktio(ODP_QUEUE_TYPE_SCHED, pkt_pool, true); CU_ASSERT_FATAL(pktio != ODP_PKTIO_INVALID); @@ -262,7 +262,7 @@ static void _classification_test_pmr_term_tcp_dport(int num_pkt) odp_pktio_mac_addr(pktio, eth->dst.addr, ODPH_ETHADDR_LEN); tcp = (odph_tcphdr_t *)odp_packet_l4_ptr(pkt, NULL); - tcp->dst_port = odp_cpu_to_be_16(CLS_DEFAULT_DPORT); + tcp->dst_port = val; enqueue_pktio_interface(pkt, pktio); } @@ -320,7 +320,7 @@ static void _classification_test_pmr_term_tcp_dport(int num_pkt) if ((i % 5) < 2) { sent_queue++; - tcp->dst_port = odp_cpu_to_be_16(CLS_DEFAULT_DPORT); + tcp->dst_port = val; } else { sent_default++; tcp->dst_port = odp_cpu_to_be_16(CLS_DEFAULT_DPORT + 1); @@ -341,8 +341,7 @@ static void _classification_test_pmr_term_tcp_dport(int num_pkt) if (retqueue == queue) { recv_queue++; - CU_ASSERT(tcp->dst_port == - odp_cpu_to_be_16(CLS_DEFAULT_DPORT)); + CU_ASSERT(tcp->dst_port == val); } else if (retqueue == default_queue) { recv_default++; CU_ASSERT(tcp->dst_port == @@ -388,8 +387,8 @@ static void classification_test_pmr_term_tcp_sport(void) odp_pmr_param_t pmr_param; odph_ethhdr_t *eth; - val = CLS_DEFAULT_SPORT; - mask = 0xffff; + val = odp_cpu_to_be_16(CLS_DEFAULT_SPORT); + mask = odp_cpu_to_be_16(0xffff); seqno = 0; pktio = create_pktio(ODP_QUEUE_TYPE_SCHED, pkt_pool, true); @@ -433,7 +432,7 @@ static void classification_test_pmr_term_tcp_sport(void) odp_pktio_mac_addr(pktio, eth->dst.addr, ODPH_ETHADDR_LEN); tcp = (odph_tcphdr_t *)odp_packet_l4_ptr(pkt, NULL); - tcp->src_port = odp_cpu_to_be_16(CLS_DEFAULT_SPORT); + tcp->src_port = val; enqueue_pktio_interface(pkt, pktio); @@ -501,8 +500,8 @@ static void classification_test_pmr_term_udp_dport(void) odph_ethhdr_t *eth; cls_packet_info_t pkt_info; - val = CLS_DEFAULT_DPORT; - mask = 0xffff; + val = odp_cpu_to_be_16(CLS_DEFAULT_DPORT); + mask = odp_cpu_to_be_16(0xffff); seqno = 0; pktio = create_pktio(ODP_QUEUE_TYPE_SCHED, pkt_pool, true); @@ -548,7 +547,7 @@ static void classification_test_pmr_term_udp_dport(void) odp_pktio_mac_addr(pktio, eth->dst.addr, ODPH_ETHADDR_LEN); udp = (odph_udphdr_t *)odp_packet_l4_ptr(pkt, NULL); - udp->dst_port = odp_cpu_to_be_16(CLS_DEFAULT_DPORT); + udp->dst_port = val; enqueue_pktio_interface(pkt, pktio); @@ -617,8 +616,8 @@ static void classification_test_pmr_term_udp_sport(void) odph_ethhdr_t *eth; cls_packet_info_t pkt_info; - val = CLS_DEFAULT_SPORT; - mask = 0xffff; + val = odp_cpu_to_be_16(CLS_DEFAULT_SPORT); + mask = odp_cpu_to_be_16(0xffff); seqno = 0; pktio = create_pktio(ODP_QUEUE_TYPE_SCHED, pkt_pool, true); @@ -664,7 +663,7 @@ static void classification_test_pmr_term_udp_sport(void) odp_pktio_mac_addr(pktio, eth->dst.addr, ODPH_ETHADDR_LEN); udp = (odph_udphdr_t *)odp_packet_l4_ptr(pkt, NULL); - udp->src_port = odp_cpu_to_be_16(CLS_DEFAULT_SPORT); + udp->src_port = val; enqueue_pktio_interface(pkt, pktio); diff --git a/test/validation/api/classification/odp_classification_tests.c b/test/validation/api/classification/odp_classification_tests.c index 9df869ece..3a97023cf 100644 --- a/test/validation/api/classification/odp_classification_tests.c +++ b/test/validation/api/classification/odp_classification_tests.c @@ -223,8 +223,8 @@ void configure_cls_pmr_chain(void) cos_list[CLS_PMR_CHAIN_SRC]); CU_ASSERT_FATAL(pmr_list[CLS_PMR_CHAIN_SRC] != ODP_PMR_INVALID); - val = CLS_PMR_CHAIN_PORT; - maskport = 0xffff; + val = odp_cpu_to_be_16(CLS_PMR_CHAIN_PORT); + maskport = odp_cpu_to_be_16(0xffff); odp_cls_pmr_param_init(&pmr_param); pmr_param.term = find_first_supported_l3_pmr(); pmr_param.match.value = &val; @@ -573,8 +573,8 @@ void configure_pmr_cos(void) cos_list[CLS_PMR] = odp_cls_cos_create(cosname, &cls_param); CU_ASSERT_FATAL(cos_list[CLS_PMR] != ODP_COS_INVALID); - val = CLS_PMR_PORT; - mask = 0xffff; + val = odp_cpu_to_be_16(CLS_PMR_PORT); + mask = odp_cpu_to_be_16(0xffff); odp_cls_pmr_param_init(&pmr_param); pmr_param.term = find_first_supported_l3_pmr(); pmr_param.match.value = &val; @@ -655,8 +655,8 @@ void configure_pktio_pmr_composite(void) pmr_params[0].match.mask = &mask; pmr_params[0].val_sz = sizeof(addr); - val = CLS_PMR_SET_PORT; - maskport = 0xffff; + val = odp_cpu_to_be_16(CLS_PMR_SET_PORT); + maskport = odp_cpu_to_be_16(0xffff); odp_cls_pmr_param_init(&pmr_params[1]); pmr_params[1].term = find_first_supported_l3_pmr(); pmr_params[1].match.value = &val; commit 6f855d843b1d87013908998a050e16c435b89e07 Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Thu Oct 24 15:10:01 2019 +0300 linux-gen: cls: change ODP_PMR_DMAC to big endian Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Stanislaw Kardach <skardach(a)marvell.com> diff --git a/platform/linux-generic/odp_classification.c b/platform/linux-generic/odp_classification.c index 1d717f413..9ad3380e5 100644 --- a/platform/linux-generic/odp_classification.c +++ b/platform/linux-generic/odp_classification.c @@ -921,24 +921,23 @@ static inline int verify_pmr_dmac(const uint8_t *pkt_addr, odp_packet_hdr_t *pkt_hdr, pmr_term_value_t *term_value) { - uint64_t dmac = 0; - uint64_t dmac_be = 0; const _odp_ethhdr_t *eth; + uint16_t dmac[3]; + uint16_t *mask = (uint16_t *)&term_value->match.mask; + uint16_t *value = (uint16_t *)&term_value->match.value; if (!packet_hdr_has_eth(pkt_hdr)) return 0; eth = (const _odp_ethhdr_t *)(pkt_addr + pkt_hdr->p.l2_offset); - memcpy(&dmac_be, eth->dst.addr, _ODP_ETHADDR_LEN); - dmac = odp_be_to_cpu_64(dmac_be); - /* since we are converting a 48 bit ethernet address from BE to cpu - format using odp_be_to_cpu_64() the last 16 bits needs to be right - shifted */ - if (dmac_be != dmac) - dmac = dmac >> (64 - (_ODP_ETHADDR_LEN * 8)); - - if (term_value->match.value == (dmac & term_value->match.mask)) + memcpy(dmac, eth->dst.addr, _ODP_ETHADDR_LEN); + dmac[0] &= mask[0]; + dmac[1] &= mask[1]; + dmac[2] &= mask[2]; + + if (value[0] == dmac[0] && value[1] == dmac[1] && value[2] == dmac[2]) return 1; + return 0; } diff --git a/test/validation/api/classification/classification.h b/test/validation/api/classification/classification.h index ae8d57221..c16b340c9 100644 --- a/test/validation/api/classification/classification.h +++ b/test/validation/api/classification/classification.h @@ -19,8 +19,8 @@ #define CLS_DEFAULT_DADDR "10.0.0.100/32" #define CLS_DEFAULT_SPORT 1024 #define CLS_DEFAULT_DPORT 2048 -#define CLS_DEFAULT_DMAC 0x010203040506 -#define CLS_DEFAULT_SMAC 0x060504030201 +#define CLS_DEFAULT_DMAC {0x01, 0x02, 0x03, 0x04, 0x05, 0x06} +#define CLS_DEFAULT_SMAC {0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c} /* Config values for Error CoS */ #define TEST_ERROR 1 diff --git a/test/validation/api/classification/odp_classification_common.c b/test/validation/api/classification/odp_classification_common.c index ad64b9d4a..851f949cc 100644 --- a/test/validation/api/classification/odp_classification_common.c +++ b/test/validation/api/classification/odp_classification_common.c @@ -240,9 +240,6 @@ odp_packet_t create_packet(cls_packet_info_t pkt_info) odph_ipv4hdr_t *ip; odph_ipv6hdr_t *ipv6; uint16_t payload_len; - uint64_t src_mac = CLS_DEFAULT_SMAC; - uint64_t dst_mac = CLS_DEFAULT_DMAC; - uint64_t dst_mac_be; uint32_t addr = 0; uint32_t mask; odp_packet_t pkt; @@ -257,12 +254,8 @@ odp_packet_t create_packet(cls_packet_info_t pkt_info) uint16_t eth_type; odp_u16be_t *vlan_type; odph_vlanhdr_t *vlan_hdr; - - /* 48 bit ethernet address needs to be left shifted for proper - value after changing to be*/ - dst_mac_be = odp_cpu_to_be_64(dst_mac); - if (dst_mac != dst_mac_be) - dst_mac_be = dst_mac_be >> (64 - 8 * ODPH_ETHADDR_LEN); + uint8_t src_mac[] = CLS_DEFAULT_SMAC; + uint8_t dst_mac[] = CLS_DEFAULT_DMAC; payload_len = sizeof(cls_test_packet_t) + pkt_info.len; seqno = odp_atomic_fetch_inc_u32(pkt_info.seq); @@ -287,7 +280,7 @@ odp_packet_t create_packet(cls_packet_info_t pkt_info) odp_packet_l2_offset_set(pkt, 0); ethhdr = (odph_ethhdr_t *)odp_packet_l2_ptr(pkt, NULL); memcpy(ethhdr->src.addr, &src_mac, ODPH_ETHADDR_LEN); - memcpy(ethhdr->dst.addr, &dst_mac_be, ODPH_ETHADDR_LEN); + memcpy(ethhdr->dst.addr, &dst_mac, ODPH_ETHADDR_LEN); vlan_type = (odp_u16be_t *)(void *)&ethhdr->type; vlan_hdr = (odph_vlanhdr_t *)(ethhdr + 1); diff --git a/test/validation/api/classification/odp_classification_test_pmr.c b/test/validation/api/classification/odp_classification_test_pmr.c index abb1a743b..9ac735020 100644 --- a/test/validation/api/classification/odp_classification_test_pmr.c +++ b/test/validation/api/classification/odp_classification_test_pmr.c @@ -821,8 +821,6 @@ static void classification_test_pmr_term_dmac(void) { odp_packet_t pkt; uint32_t seqno; - uint64_t val; - uint64_t mask; int retval; odp_pktio_t pktio; odp_queue_t queue; @@ -839,9 +837,8 @@ static void classification_test_pmr_term_dmac(void) odp_pmr_param_t pmr_param; odph_ethhdr_t *eth; cls_packet_info_t pkt_info; - - val = CLS_DEFAULT_DMAC; /* 48 bit Ethernet Mac address */ - mask = 0xffffffffffff; + uint8_t val[] = {0x99, 0xaa, 0xbb, 0xcc, 0xdd, 0xee}; + uint8_t mask[] = {0xff, 0xff, 0xff, 0xff, 0xff, 0xff}; seqno = 0; pktio = create_pktio(ODP_QUEUE_TYPE_SCHED, pkt_pool, true); @@ -880,6 +877,8 @@ static void classification_test_pmr_term_dmac(void) pkt_info.udp = true; pkt = create_packet(pkt_info); CU_ASSERT_FATAL(pkt != ODP_PACKET_INVALID); + eth = (odph_ethhdr_t *)odp_packet_l2_ptr(pkt, NULL); + memcpy(eth->dst.addr, val, ODPH_ETHADDR_LEN); seqno = cls_pkt_get_seq(pkt); CU_ASSERT(seqno != TEST_SEQ_INVALID); @@ -896,9 +895,6 @@ static void classification_test_pmr_term_dmac(void) /* Other packets delivered to default queue */ pkt = create_packet(default_pkt_info); CU_ASSERT_FATAL(pkt != ODP_PACKET_INVALID); - eth = (odph_ethhdr_t *)odp_packet_l2_ptr(pkt, NULL); - memset(eth->dst.addr, 0, ODPH_ETHADDR_LEN); - CU_ASSERT_FATAL(pkt != ODP_PACKET_INVALID); seqno = cls_pkt_get_seq(pkt); CU_ASSERT(seqno != TEST_SEQ_INVALID); commit 35a6b2e5c369e81196f6c288085990eaaf83f360 Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Thu Oct 24 14:41:59 2019 +0300 linux-gen: cls: change ODP_PMR_VLAN_ID_X to big endian Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Stanislaw Kardach <skardach(a)marvell.com> diff --git a/platform/linux-generic/odp_classification.c b/platform/linux-generic/odp_classification.c index 308f060b1..1d717f413 100644 --- a/platform/linux-generic/odp_classification.c +++ b/platform/linux-generic/odp_classification.c @@ -1033,8 +1033,8 @@ static inline int verify_pmr_vlan_id_x(const uint8_t *pkt_addr, if (pkt_hdr->p.input_flags.vlan_qinq) vlan++; - tci = odp_be_to_cpu_16(vlan->tci); - vlan_id = tci & 0x0fff; + tci = vlan->tci; + vlan_id = tci & odp_cpu_to_be_16(0x0fff); if (term_value->match.value == (vlan_id & term_value->match.mask)) return 1; diff --git a/test/validation/api/classification/odp_classification_test_pmr.c b/test/validation/api/classification/odp_classification_test_pmr.c index 7111595c5..abb1a743b 100644 --- a/test/validation/api/classification/odp_classification_test_pmr.c +++ b/test/validation/api/classification/odp_classification_test_pmr.c @@ -1170,8 +1170,8 @@ static void classification_test_pmr_term_vlan_id_x(void) odph_vlanhdr_t *vlan_x; cls_packet_info_t pkt_info; - val = 0x345; - mask = 0xfff; + val = odp_cpu_to_be_16(0x345); + mask = odp_cpu_to_be_16(0xfff); seqno = 0; pktio = create_pktio(ODP_QUEUE_TYPE_SCHED, pkt_pool, true); @@ -1218,7 +1218,7 @@ static void classification_test_pmr_term_vlan_id_x(void) odp_pktio_mac_addr(pktio, eth->dst.addr, ODPH_ETHADDR_LEN); vlan_x = (odph_vlanhdr_t *)(eth + 1); vlan_x++; - vlan_x->tci = odp_cpu_to_be_16(val); + vlan_x->tci = val; enqueue_pktio_interface(pkt, pktio); pkt = receive_packet(&retqueue, ODP_TIME_SEC_IN_NS); commit 0e2ee113fcc0e7383d9e465c11a8495a65f7a171 Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Thu Oct 24 14:38:47 2019 +0300 linux-gen: cls: change ODP_PMR_VLAN_ID_0 to big endian Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Stanislaw Kardach <skardach(a)marvell.com> diff --git a/platform/linux-generic/odp_classification.c b/platform/linux-generic/odp_classification.c index 91d04640f..308f060b1 100644 --- a/platform/linux-generic/odp_classification.c +++ b/platform/linux-generic/odp_classification.c @@ -1006,8 +1006,8 @@ static inline int verify_pmr_vlan_id_0(const uint8_t *pkt_addr, eth = (const _odp_ethhdr_t *)(pkt_addr + pkt_hdr->p.l2_offset); vlan = (const _odp_vlanhdr_t *)(eth + 1); - tci = odp_be_to_cpu_16(vlan->tci); - vlan_id = tci & 0x0fff; + tci = vlan->tci; + vlan_id = tci & odp_cpu_to_be_16(0x0fff); if (term_value->match.value == (vlan_id & term_value->match.mask)) return 1; diff --git a/test/validation/api/classification/odp_classification_test_pmr.c b/test/validation/api/classification/odp_classification_test_pmr.c index 5dd16c321..7111595c5 100644 --- a/test/validation/api/classification/odp_classification_test_pmr.c +++ b/test/validation/api/classification/odp_classification_test_pmr.c @@ -1059,8 +1059,8 @@ static void classification_test_pmr_term_vlan_id_0(void) odph_vlanhdr_t *vlan_0; cls_packet_info_t pkt_info; - val = 0x123; - mask = 0xfff; + val = odp_cpu_to_be_16(0x123); + mask = odp_cpu_to_be_16(0xfff); seqno = 0; pktio = create_pktio(ODP_QUEUE_TYPE_SCHED, pkt_pool, true); @@ -1105,7 +1105,7 @@ static void classification_test_pmr_term_vlan_id_0(void) odp_pktio_mac_addr(pktio, eth->src.addr, ODPH_ETHADDR_LEN); odp_pktio_mac_addr(pktio, eth->dst.addr, ODPH_ETHADDR_LEN); vlan_0 = (odph_vlanhdr_t *)(eth + 1); - vlan_0->tci = odp_cpu_to_be_16(val); + vlan_0->tci = val; enqueue_pktio_interface(pkt, pktio); pkt = receive_packet(&retqueue, ODP_TIME_SEC_IN_NS); commit 0cfdb2c5a6b3afe0e4b0eee2b3749a0cfca3917d Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Thu Oct 24 14:27:09 2019 +0300 linux-gen: cls: change ODP_PMR_ETHTYPE_X to big endian Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Stanislaw Kardach <skardach(a)marvell.com> diff --git a/platform/linux-generic/odp_classification.c b/platform/linux-generic/odp_classification.c index e640ede2e..91d04640f 100644 --- a/platform/linux-generic/odp_classification.c +++ b/platform/linux-generic/odp_classification.c @@ -1132,7 +1132,7 @@ static inline int verify_pmr_eth_type_x(const uint8_t *pkt_addr, if (pkt_hdr->p.input_flags.vlan_qinq) vlan++; - ethtype = odp_be_to_cpu_16(vlan->type); + ethtype = vlan->type; if (term_value->match.value == (ethtype & term_value->match.mask)) return 1; diff --git a/test/validation/api/classification/odp_classification_test_pmr.c b/test/validation/api/classification/odp_classification_test_pmr.c index abcc73e07..5dd16c321 100644 --- a/test/validation/api/classification/odp_classification_test_pmr.c +++ b/test/validation/api/classification/odp_classification_test_pmr.c @@ -1391,8 +1391,8 @@ static void classification_test_pmr_term_eth_type_x(void) odph_vlanhdr_t *vlan_x; cls_packet_info_t pkt_info; - val = 0x0800; - mask = 0xffff; + val = odp_cpu_to_be_16(0x0800); + mask = odp_cpu_to_be_16(0xffff); seqno = 0; pktio = create_pktio(ODP_QUEUE_TYPE_SCHED, pkt_pool, true); @@ -1440,7 +1440,7 @@ static void classification_test_pmr_term_eth_type_x(void) vlan_x = (odph_vlanhdr_t *)(eth + 1); vlan_x++; vlan_x->tci = odp_cpu_to_be_16(0x123); - vlan_x->type = odp_cpu_to_be_16(val); + vlan_x->type = val; enqueue_pktio_interface(pkt, pktio); commit b7a8fd47362a3853db8cc3ea56fe1d268defa574 Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Thu Oct 24 13:55:29 2019 +0300 linux-gen: cls: change ODP_PMR_ETHTYPE_0 to big endian Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Stanislaw Kardach <skardach(a)marvell.com> diff --git a/platform/linux-generic/odp_classification.c b/platform/linux-generic/odp_classification.c index 0119f7537..e640ede2e 100644 --- a/platform/linux-generic/odp_classification.c +++ b/platform/linux-generic/odp_classification.c @@ -1107,7 +1107,7 @@ static inline int verify_pmr_eth_type_0(const uint8_t *pkt_addr, return 0; eth = (const _odp_ethhdr_t *)(pkt_addr + pkt_hdr->p.l2_offset); - ethtype = odp_be_to_cpu_16(eth->type); + ethtype = eth->type; if (term_value->match.value == (ethtype & term_value->match.mask)) return 1; diff --git a/test/validation/api/classification/odp_classification_test_pmr.c b/test/validation/api/classification/odp_classification_test_pmr.c index 44942de98..abcc73e07 100644 --- a/test/validation/api/classification/odp_classification_test_pmr.c +++ b/test/validation/api/classification/odp_classification_test_pmr.c @@ -1282,8 +1282,8 @@ static void classification_test_pmr_term_eth_type_0(void) odph_ethhdr_t *eth; cls_packet_info_t pkt_info; - val = ODPH_ETHTYPE_IPV6; - mask = 0xffff; + val = odp_cpu_to_be_16(ODPH_ETHTYPE_IPV6); + mask = odp_cpu_to_be_16(0xffff); seqno = 0; pktio = create_pktio(ODP_QUEUE_TYPE_SCHED, pkt_pool, true); commit 0df7a38e2b9be4ad72528d43551e56ac4d152b96 Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Wed Oct 23 17:21:13 2019 +0300 linux-gen: cls: check PMR value size Check that PMR value size matches API specification. Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Stanislaw Kardach <skardach(a)marvell.com> diff --git a/platform/linux-generic/odp_classification.c b/platform/linux-generic/odp_classification.c index 907d68481..0119f7537 100644 --- a/platform/linux-generic/odp_classification.c +++ b/platform/linux-generic/odp_classification.c @@ -568,12 +568,77 @@ int odp_cos_with_l3_qos(odp_pktio_t pktio_in, return 0; } -static int odp_pmr_create_term(pmr_term_value_t *value, - const odp_pmr_param_t *param) +static int pmr_create_term(pmr_term_value_t *value, + const odp_pmr_param_t *param) { - value->term = param->term; - value->range_term = param->range_term; + uint32_t size; uint8_t i; + int custom = 0; + odp_cls_pmr_term_t term = param->term; + + value->term = term; + value->range_term = param->range_term; + + switch (term) { + case ODP_PMR_IPPROTO: + size = 1; + break; + + case ODP_PMR_ETHTYPE_0: + /* Fall through */ + case ODP_PMR_ETHTYPE_X: + /* Fall through */ + case ODP_PMR_VLAN_ID_0: + /* Fall through */ + case ODP_PMR_VLAN_ID_X: + /* Fall through */ + case ODP_PMR_UDP_DPORT: + /* Fall through */ + case ODP_PMR_TCP_DPORT: + /* Fall through */ + case ODP_PMR_UDP_SPORT: + /* Fall through */ + case ODP_PMR_TCP_SPORT: + size = 2; + break; + + case ODP_PMR_LEN: + /* Fall through */ + case ODP_PMR_SIP_ADDR: + /* Fall through */ + case ODP_PMR_DIP_ADDR: + /* Fall through */ + case ODP_PMR_IPSEC_SPI: + /* Fall through */ + case ODP_PMR_LD_VNI: + size = 4; + break; + + case ODP_PMR_DMAC: + size = 6; + break; + + case ODP_PMR_SIP6_ADDR: + /* Fall through */ + case ODP_PMR_DIP6_ADDR: + size = 16; + break; + + case ODP_PMR_CUSTOM_FRAME: + custom = 1; + size = CLS_PMR_TERM_BYTES_MAX; + break; + + default: + ODP_ERR("Bad PRM term\n"); + return -1; + } + + if ((!custom && param->val_sz != size) || + (custom && param->val_sz > size)) { + ODP_ERR("Bad PMR value size: %u\n", param->val_sz); + return -1; + } switch (value->term) { case ODP_PMR_SIP6_ADDR: @@ -655,7 +720,6 @@ odp_pmr_t odp_cls_pmr_create(const odp_pmr_param_t *terms, int num_terms, pmr_t *pmr; int i; odp_pmr_t id; - int val_sz; uint32_t loc; cos_t *cos_src = get_cos_entry(src_cos); cos_t *cos_dst = get_cos_entry(dst_cos); @@ -680,13 +744,8 @@ odp_pmr_t odp_cls_pmr_create(const odp_pmr_param_t *terms, int num_terms, pmr->s.num_pmr = num_terms; for (i = 0; i < num_terms; i++) { - val_sz = terms[i].val_sz; - if (val_sz > CLS_PMR_TERM_BYTES_MAX) { + if (pmr_create_term(&pmr->s.pmr_term_value[i], &terms[i])) { pmr->s.valid = 0; - return ODP_PMR_INVALID; - } - if (0 > odp_pmr_create_term(&pmr->s.pmr_term_value[i], - &terms[i])) { UNLOCK(&pmr->s.lock); return ODP_PMR_INVALID; } diff --git a/test/validation/api/classification/odp_classification_test_pmr.c b/test/validation/api/classification/odp_classification_test_pmr.c index a01c0a403..44942de98 100644 --- a/test/validation/api/classification/odp_classification_test_pmr.c +++ b/test/validation/api/classification/odp_classification_test_pmr.c @@ -927,8 +927,8 @@ static void classification_test_pmr_term_packet_len(void) { odp_packet_t pkt; uint32_t seqno; - uint16_t val; - uint16_t mask; + uint32_t val; + uint32_t mask; int retval; odp_pktio_t pktio; odp_queue_t queue; commit 139b28399d3bd1f53624908c9fd99c1f14283cdd Author: Petri Savolainen <petri.savolainen(a)nokia.com> Date: Wed Oct 23 16:36:40 2019 +0300 api: cls: PMR value in big endian format PMR term documentation was not explicit about endianness of value and mask fields. Most terms assumed CPU endian, but terms with larger data sizes assumed big endian (MAC, IPv6 address and custom). Harmonize term specification so that all terms expect value/mask data to be in big endian format, have fixed size and allow free memory alignment. Packet length term is an exception to this as it does not represent a field in a packet. Signed-off-by: Petri Savolainen <petri.savolainen(a)nokia.com> Reviewed-by: Matias Elo <matias.elo(a)nokia.com> Reviewed-by: Stanislaw Kardach <skardach(a)marvell.com> diff --git a/include/odp/api/spec/classification.h b/include/odp/api/spec/classification.h index 043059fa1..67fdd20aa 100644 --- a/include/odp/api/spec/classification.h +++ b/include/odp/api/spec/classification.h @@ -461,44 +461,92 @@ int odp_cos_with_l3_qos(odp_pktio_t pktio_in, */ /** - * Packet Matching Rule field enumeration - * for fields that may be used to calculate - * the PMR, if present in a packet. + * Packet Matching Rule terms + * + * This enumeration selects the protocol field that is matched against PMR + * value/mask or value range. Protocol field values and masks are passed in big + * endian (network endian) format. However, ODP_PMR_LEN value and range are + * passed in CPU native endian (uint32_t words), as the term does not represent + * a protocol field. + * + * PMR value/mask data size is term specific. This size must be set into val_sz + * field of odp_pmr_param_t. There is no alignment requirement for PMR + * value/mask data. */ typedef enum { - ODP_PMR_LEN, /**< Total length of received packet*/ - ODP_PMR_ETHTYPE_0, /**< Initial (outer) - Ethertype only (*val=uint16_t)*/ - ODP_PMR_ETHTYPE_X, /**< Ethertype of most inner VLAN tag - (*val=uint16_t)*/ - ODP_PMR_VLAN_ID_0, /**< First VLAN ID (outer) (*val=uint16_t) */ - ODP_PMR_VLAN_ID_X, /**< Last VLAN ID (inner) (*val=uint16_t) */ - ODP_PMR_DMAC, /**< destination MAC address (*val=uint64_t)*/ - ODP_PMR_IPPROTO, /**< IP Protocol or IPv6 Next Header - (*val=uint8_t) */ - ODP_PMR_UDP_DPORT, /**< Destination UDP port, implies IPPROTO=17*/ - ODP_PMR_TCP_DPORT, /**< Destination TCP port implies IPPROTO=6*/ - ODP_PMR_UDP_SPORT, /**< Source UDP Port (*val=uint16_t)*/ - ODP_PMR_TCP_SPORT, /**< Source TCP port (*val=uint16_t)*/ - ODP_PMR_SIP_ADDR, /**< Source IP address (uint32_t)*/ - ODP_PMR_DIP_ADDR, /**< Destination IP address (uint32_t)*/ - ODP_PMR_SIP6_ADDR, /**< Source IP address (uint8_t[16])*/ - ODP_PMR_DIP6_ADDR, /**< Destination IP address (uint8_t[16])*/ - ODP_PMR_IPSEC_SPI, /**< IPsec session identifier(*val=uint32_t)*/ - ODP_PMR_LD_VNI, /**< NVGRE/VXLAN network identifier - (*val=uint32_t)*/ - ODP_PMR_CUSTOM_FRAME, /**< Custom match rule, offset from start of - frame. The match is defined by the offset, the - expected value, and its size. They must be - applied before any other PMR. - (*val=uint8_t[val_sz])*/ + /** Total length of received packet. Exceptionally, value and mask are + * uint32_t (val_sz = 4) in CPU endian. */ + ODP_PMR_LEN, + + /** Initial (outer) Ethertype only (val_sz = 2) */ + ODP_PMR_ETHTYPE_0, + + /** Ethertype of most inner VLAN tag (val_sz = 2) */ + ODP_PMR_ETHTYPE_X, + + /** First (outer) VLAN ID (val_sz = 2) */ + ODP_PMR_VLAN_ID_0, + + /** Last (most inner) VLAN ID (val_sz = 2) */ + ODP_PMR_VLAN_ID_X, + + /** Destination MAC address (val_sz = 6) */ + ODP_PMR_DMAC, + + /** IPv4 Protocol or IPv6 Next Header (val_sz = 1) */ + ODP_PMR_IPPROTO, + + /** Destination UDP port (val_sz = 2) */ + ODP_PMR_UDP_DPORT, + + /** Destination TCP port (val_sz = 2) */ + ODP_PMR_TCP_DPORT, + + /** Source UDP port (val_sz = 2) */ + ODP_PMR_UDP_SPORT, + + /** Source TCP port (val_sz = 2) */ + ODP_PMR_TCP_SPORT, + + /** Source IPv4 address (val_sz = 4) */ + ODP_PMR_SIP_ADDR, + + /** Destination IPv4 address (val_sz = 4) */ + ODP_PMR_DIP_ADDR, + + /** Source IPv6 address (val_sz = 16) */ + ODP_PMR_SIP6_ADDR, + + /** Destination IPv6 address (val_sz = 16) */ + ODP_PMR_DIP6_ADDR, + + /** IPsec session identifier (val_sz = 4)*/ + ODP_PMR_IPSEC_SPI, + + /** NVGRE/VXLAN network identifier (val_sz = 4) */ + ODP_PMR_LD_VNI, + + /** + * Custom frame match rule + * + * PMR offset is counted from the start of the packet. The match is + * defined by the offset, the expected value, and its size. Custom frame + * rules must be applied before any other PMR. + */ + ODP_PMR_CUSTOM_FRAME, /** Inner header may repeat above values with this offset */ ODP_PMR_INNER_HDR_OFF = 32 + } odp_cls_pmr_term_t; /** * Packet Matching Rule parameter structure + * + * Match value/mask size and endianness are defined in PMR term documentation + * (@see odp_cls_pmr_term_t). Most values and masks are passed in big + * endian format without data alignment requirement. ODP_PMR_LEN is + * an exception to this (uint32_t in CPU endian). */ typedef struct odp_pmr_param_t { odp_cls_pmr_term_t term; /**< Packet Matching Rule term */ @@ -510,21 +558,24 @@ typedef struct odp_pmr_param_t { union { /** Parameters for single-valued matches */ struct { - /** Value to be matched */ - const void *value; - - /** Masked set of bits to be matched */ - const void *mask; + /** Points to the value to be matched. Value size and + * endianness are defined by the term used. Values of + * protocol fields are defined in big endian format. + */ + const void *value; + + /** Mask of the bits to be matched. The same size and + * endianness is used than with the value. */ + const void *mask; } match; /** Parameter for range value matches */ struct { - /** Start and End values are included in the range */ - /** start value of range */ - const void *val_start; + /** Start value of the range */ + const void *val_start; - /** End value of range */ - const void *val_end; + /** End value of the range */ + const void *val_end; } range; }; uint32_t val_sz; /**< Size of the term value */ ----------------------------------------------------------------------- Summary of changes: configure.ac | 2 +- example/classifier/odp_classifier.c | 224 +++++++++--------- include/odp/api/spec/classification.h | 172 +++++++++----- include/odp/api/spec/packet_io.h | 14 +- include/odp/api/spec/pool.h | 8 +- include/odp/api/spec/shared_memory.h | 2 +- include/odp/api/spec/version.h.in | 1 - .../include/odp_classification_datamodel.h | 79 ++++--- platform/linux-generic/odp_classification.c | 249 +++++++++++++-------- .../validation/api/classification/classification.h | 4 +- .../api/classification/odp_classification_common.c | 13 +- .../classification/odp_classification_test_pmr.c | 236 +++++++++++++++---- .../api/classification/odp_classification_tests.c | 18 +- 13 files changed, 678 insertions(+), 344 deletions(-) hooks/post-receive --

6 years, 10 months

← Newer
1
...
4
5
6
7
8
9
10
...
124
Older →

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

git-lng-odp