CoreSight currently models exception entry by changing the preceding instruction range into a taken branch. This can give an IRQ the source PC of an instruction that already retired and overwrite a real branch immediately before the exception.
For an untaken B.LS followed by an IRQ, perf script currently reports:
hw int 4000f8 => ffff800080010c80 b.ls #0x400118 ... iret ffff800080012284 => 4000fc eret
The hardware trace supplies 0x4000fc as the preferred return address. For an IRQ, this is the architectural resume PC (Arm ARM, R_VBQMV). B.LS has retired, and the saved PC identifies the boundary before the following MOV. Using this PC as the IRQ source reflects the architectural state at exception entry:
hw int 4000fc => ffff800080010c80 movz x2, #0x1796 ... iret ffff800080012284 => 4000fc eret
The IRQ sample represents the transfer from this architectural execution position to the handler. The series synthesizes exception entries from exception packets, preserving the preceding branch and its outcome.
The supporting changes:
- Let decoders supply sample.ret_addr so later instruction fetching cannot change thread-stack return addresses. Apply this to Intel PT asynchronous samples as well. - Prepare packet ISA and instruction-size handling and share sample synthesis helpers. Mark untaken branches and break history when instruction memory is unavailable. - Add a thread-stack regression test and four AArch64 CoreSight tests. IRQs and page faults must resume at the entry PC; SVC and emulated MRS must resume four bytes later.
Based on the AI search and test on my x86 machine, this matches perf's Intel PT handling of IRQs. Intel PT records the next instruction's IP in the FUP packet, and Perf uses that IP as the interrupt sample's source.
This series is verified on Orion6 board with "perf test coresight".
Signed-off-by: Leo Yan leo.yan@arm.com --- Changes in v2:
- Rework the fix around exception packets to preserve both exception entries and preceding branches. - Add explicit return addresses and the Intel PT asynchronous-branch fix. - Split out packet/synthesis preparation, record not-taken branches and handle unreadable instruction memory. - Add thread-stack regression coverage and four CoreSight tests. - Link to v1: https://lore.kernel.org/r/20260713-perf_cs_etm_fix_non_taken-v1-0-4561607fc6...
--- Leo Yan (14): perf sample: Allow decoders to supply branch return addresses perf intel-pt: Preserve return addresses for asynchronous branches perf cs-etm: Break branch history when instruction memory is unavailable perf cs-etm: Centralize packet ISA initialization perf cs-etm: Use the recorded instruction size for A32 and A64 perf cs-etm: Mark branches that were not taken perf cs-etm: Factor out final instruction sample synthesis perf cs-etm: Centralize branch sample synthesis checks perf cs-etm: Classify exception calls using the exception packet perf cs-etm: Synthesize exception entries separately from branches perf tests: Check CoreSight IRQ entry and exit perf tests: Check CoreSight syscall entry and exit perf tests: Check CoreSight abort entry and exit perf tests: Check CoreSight emulated instruction entry and exit
tools/perf/tests/Build | 1 + tools/perf/tests/builtin-test.c | 3 + .../perf/tests/shell/coresight/abort_entry_exit.sh | 21 ++ tools/perf/tests/shell/coresight/irq_entry_exit.sh | 37 ++++ .../tests/shell/coresight/syscall_entry_exit.sh | 21 ++ .../perf/tests/shell/coresight/trap_entry_exit.sh | 24 +++ tools/perf/tests/shell/lib/coresight_exception.sh | 165 +++++++++++++++ tools/perf/tests/tests.h | 3 + tools/perf/tests/thread-stack.c | 106 ++++++++++ tools/perf/tests/workloads/Build | 4 + tools/perf/tests/workloads/branch_not_taken_loop.c | 33 +++ tools/perf/tests/workloads/page_fault_loop.c | 37 ++++ tools/perf/util/cs-etm-decoder/cs-etm-decoder.c | 106 +++++++--- tools/perf/util/cs-etm.c | 222 ++++++++++----------- tools/perf/util/cs-etm.h | 2 + tools/perf/util/intel-pt.c | 7 + tools/perf/util/sample.c | 1 + tools/perf/util/sample.h | 5 + tools/perf/util/thread-stack.c | 5 +- 19 files changed, 656 insertions(+), 147 deletions(-) --- base-commit: edd8a9fe2eca009599e013a29c421c7a6b5ad1b9 change-id: 20260713-perf_cs_etm_fix_non_taken-5b4d7f73f41e
Best regards,
The thread stack derives return addresses from IP + insn_len. For an interrupt or fault, the return address can instead be the sample IP even when instruction bytes at that address are available.
Add ret_addr to perf_sample and prefer it in thread_stack__trace_end() and thread_stack__process() when supplied. Initialize it to zero so other samples retain the IP + insn_len calculation.
Add a regression test for explicit exception return addresses and the ordinary call fallback when the return address is zero.
Assisted-by: Codex:gpt-6 Signed-off-by: Leo Yan leo.yan@arm.com --- tools/perf/tests/Build | 1 + tools/perf/tests/builtin-test.c | 1 + tools/perf/tests/tests.h | 1 + tools/perf/tests/thread-stack.c | 106 ++++++++++++++++++++++++++++++++++++++++ tools/perf/util/sample.c | 1 + tools/perf/util/sample.h | 5 ++ tools/perf/util/thread-stack.c | 5 +- 7 files changed, 118 insertions(+), 2 deletions(-)
diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build index 81c311b131b72715f11b6501511a18dce2af07df..d03ae938dd0ad92f78f8de6900ce7161f39fc3bc 100644 --- a/tools/perf/tests/Build +++ b/tools/perf/tests/Build @@ -29,6 +29,7 @@ perf-test-y += task-exit.o perf-test-y += sw-clock.o perf-test-y += mmap-thread-lookup.o perf-test-y += thread-maps-share.o +perf-test-y += thread-stack.o perf-test-$(CONFIG_LIBTRACEEVENT) += switch-tracking.o perf-test-y += keep-tracking.o perf-test-y += code-reading.o diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c index d2f594921e25bda9fc662e7ba82464bfef1b752b..6259ed805c5f75799d52ac742b77cf278f98bd9d 100644 --- a/tools/perf/tests/builtin-test.c +++ b/tools/perf/tests/builtin-test.c @@ -112,6 +112,7 @@ static struct test_suite *generic_tests[] = { &suite__hists_filter, &suite__mmap_thread_lookup, &suite__thread_maps_share, + &suite__thread_stack, &suite__hists_output, &suite__hists_cumulate, #ifdef HAVE_LIBTRACEEVENT diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h index 9c96f33483d1435644da6068c3802f7a914a9de0..b2520a564417b61718f1dcb23da1f8cc0f601906 100644 --- a/tools/perf/tests/tests.h +++ b/tools/perf/tests/tests.h @@ -138,6 +138,7 @@ DECLARE_SUITE(expr); DECLARE_SUITE(hists_filter); DECLARE_SUITE(mmap_thread_lookup); DECLARE_SUITE(thread_maps_share); +DECLARE_SUITE(thread_stack); DECLARE_SUITE(hists_output); DECLARE_SUITE(hists_cumulate); DECLARE_SUITE(switch_tracking); diff --git a/tools/perf/tests/thread-stack.c b/tools/perf/tests/thread-stack.c new file mode 100644 index 0000000000000000000000000000000000000000..0239e3bee34faf8fd4ddc7b723cf479609760064 --- /dev/null +++ b/tools/perf/tests/thread-stack.c @@ -0,0 +1,106 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/kernel.h> +#include <unistd.h> +#include "tests.h" +#include "util/addr_location.h" +#include "util/event.h" +#include "util/sample.h" +#include "util/thread.h" +#include "util/thread-stack.h" + +#define CALL_REF 1234UL +#define RET_REF 5678UL + +struct return_check { + unsigned int matched; + unsigned int unmatched; +}; + +static int check_call_return(struct call_return *cr, + u64 *parent_db_id __maybe_unused, void *data) +{ + struct return_check *check = data; + + if (cr->call_ref == CALL_REF && cr->return_ref == RET_REF && !cr->flags) + check->matched++; + else + check->unmatched++; + + return 0; +} + +/* A zero expected_ret_addr asks the stack to use ip + insn_len. */ +static int check_return_address(u64 expected_ret_addr, u64 actual_ret_addr, + u32 flags) +{ + struct call_return_processor *crp; + struct return_check check = { }; + struct thread *thread; + struct addr_location from = { }, to = { }; + struct perf_sample sample = { }; + int ret = TEST_FAIL; + + thread = thread__new(getpid(), getpid()); + if (!thread) + return TEST_FAIL; + + crp = call_return_processor__new(check_call_return, &check); + if (!crp) + goto out; + + sample.ip = 0x1000; /* Call or exception source addr */ + sample.addr = 0x2000; /* Callee or exception handler addr */ + sample.ret_addr = expected_ret_addr; + sample.flags = PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | flags; + sample.time = 1; + /* Model the opcode length after an instruction fetch. */ + sample.insn_len = 4; + if (thread_stack__process(thread, thread__comm(thread), &sample, + &from, &to, CALL_REF, crp)) + goto out; + + sample.ip = 0x2000; /* Return instruction addr */ + sample.addr = actual_ret_addr; /* Return branch target addr */ + sample.ret_addr = 0; + sample.flags = PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_RETURN | + (flags & (PERF_IP_FLAG_INTERRUPT | PERF_IP_FLAG_SYSCALLRET)); + sample.time = 2; + if (thread_stack__process(thread, thread__comm(thread), &sample, + &to, &from, RET_REF, crp)) + goto out; + + if (check.matched == 1 && !check.unmatched) + ret = TEST_OK; + +out: + thread__put(thread); + call_return_processor__free(crp); + return ret; +} + +static int test__thread_stack(struct test_suite *test __maybe_unused, + int subtest __maybe_unused) +{ + static const struct { + const char *name; + u64 expected_ret_addr; + u64 actual_ret_addr; + u32 flags; + } cases[] = { + { "ordinary call", 0, 0x1004, 0 }, + { "interrupt", 0x1000, 0x1000, PERF_IP_FLAG_ASYNC | PERF_IP_FLAG_INTERRUPT }, + { "fault or trap", 0x1000, 0x1000, PERF_IP_FLAG_INTERRUPT }, + { "SVC", 0x1004, 0x1004, PERF_IP_FLAG_SYSCALLRET }, + }; + + for (size_t i = 0; i < ARRAY_SIZE(cases); i++) { + if (check_return_address(cases[i].expected_ret_addr, + cases[i].actual_ret_addr, cases[i].flags)) { + pr_debug("Incorrect return address for %s\n", cases[i].name); + return TEST_FAIL; + } + } + return TEST_OK; +} + +DEFINE_SUITE("Thread stack return addresses after instruction fetching", thread_stack); diff --git a/tools/perf/util/sample.c b/tools/perf/util/sample.c index bccc19e2aaf25118a8ecde88473aa6cb16fa561a..4abb689132bc27d0f2318eeafca20f7e57621837 100644 --- a/tools/perf/util/sample.c +++ b/tools/perf/util/sample.c @@ -29,6 +29,7 @@ void perf_sample__init(struct perf_sample *sample, bool all) sample->intr_regs = NULL; sample->merged_callchain = false; sample->callchain = NULL; + sample->ret_addr = 0; } }
diff --git a/tools/perf/util/sample.h b/tools/perf/util/sample.h index cb4b16654876e9a5c3bfd66be3e0235ac15795e0..865ed18200f1339f5949ef3f185ed57a34aace72 100644 --- a/tools/perf/util/sample.h +++ b/tools/perf/util/sample.h @@ -131,6 +131,11 @@ struct perf_sample { u64 time; /** @addr: The sample event PERF_SAMPLE_ADDR value. */ u64 addr; + /** + * @ret_addr: Return address supplied by the decoder for a branch sample. + * Zero means use ip + insn_len. + */ + u64 ret_addr; /** @id: The sample event PERF_SAMPLE_ID or PERF_SAMPLE_IDENTIFIER value. */ u64 id; /** @stream_id: The sample event PERF_SAMPLE_STREAM_ID value. */ diff --git a/tools/perf/util/thread-stack.c b/tools/perf/util/thread-stack.c index 1360f44421ef8bb80fe5cfdba59be7e6029b0240..d452d1a7eabb16f040ca3dada23e6fdec8fb1088 100644 --- a/tools/perf/util/thread-stack.c +++ b/tools/perf/util/thread-stack.c @@ -1030,7 +1030,7 @@ static int thread_stack__trace_end(struct thread_stack *ts, cp = call_path__findnew(cpr, ts->stack[ts->cnt - 1].cp, NULL, 0, ts->kernel_start);
- ret_addr = sample->ip + sample->insn_len; + ret_addr = sample->ret_addr ? sample->ret_addr : sample->ip + sample->insn_len;
return thread_stack__push_cp(ts, ret_addr, sample->time, ref, cp, false, true); @@ -1154,7 +1154,8 @@ int thread_stack__process(struct thread *thread, struct comm *comm, if (!sample->ip || !sample->addr) return 0;
- ret_addr = sample->ip + sample->insn_len; + /* Opcode fetching must not change a decoder-supplied return address. */ + ret_addr = sample->ret_addr ? sample->ret_addr : sample->ip + sample->insn_len; if (ret_addr == sample->addr) return 0; /* Zero-length calls are excluded */
Intel PT uses a zero instruction length for asynchronous branches so the thread stack resumes at the source IP. A dlfilter can fetch the instruction before call/return processing and replace that zero with its opcode length, advancing the saved return address.
Set sample.ret_addr to sample.ip for asynchronous branch samples so instruction fetching cannot change the return address. This also covers asynchronous trace-end samples and VM exits.
Assisted-by: Codex:gpt-6 Signed-off-by: Leo Yan leo.yan@arm.com --- tools/perf/util/intel-pt.c | 7 +++++++ 1 file changed, 7 insertions(+)
diff --git a/tools/perf/util/intel-pt.c b/tools/perf/util/intel-pt.c index 8c21c9f52d572d2dd4e15b3b5c341b253ccea1b5..e8b9c8a67b0a0635e0110eda4b6ccb2511132fc7 100644 --- a/tools/perf/util/intel-pt.c +++ b/tools/perf/util/intel-pt.c @@ -1803,6 +1803,13 @@ static int intel_pt_synth_branch_sample(struct intel_pt_queue *ptq) perf_sample__init(&sample, /*all=*/true); intel_pt_prep_b_sample(pt, ptq, event, &sample);
+ /* + * For asynchronous branches, use the sample IP as the return address + * instead of advancing it during instruction decoding. + */ + if (sample.flags & PERF_IP_FLAG_ASYNC) + sample.ret_addr = sample.ip; + sample.id = ptq->pt->branches_id; sample.stream_id = ptq->pt->branches_id;
The decoder currently ignores OCSD_GEN_TRC_ELEM_ADDR_NACC, allowing a subsequent range to be paired with stale branch history from before the unreadable code.
Treat an inaccessible instruction address as a discontinuity and flush the pending range and branch history before processing subsequent elements.
Assisted-by: Codex:gpt-6 Signed-off-by: Leo Yan leo.yan@arm.com --- tools/perf/util/cs-etm-decoder/cs-etm-decoder.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c index 26940f1f1b0bf44c8671d80020597c672c361c1a..35ca1a23e12735416d0d412a949595efc5bfe50a 100644 --- a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c +++ b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c @@ -594,7 +594,8 @@ static ocsd_datapath_resp_t cs_etm_decoder__gen_trace_elem_printer(
if (type == OCSD_GEN_TRC_ELEM_EO_TRACE || type == OCSD_GEN_TRC_ELEM_NO_SYNC || - type == OCSD_GEN_TRC_ELEM_TRACE_ON) + type == OCSD_GEN_TRC_ELEM_TRACE_ON || + type == OCSD_GEN_TRC_ELEM_ADDR_NACC) resp = cs_etm_decoder__buffer_discontinuity(etmq, packet_queue, trace_chan_id); else if (type == OCSD_GEN_TRC_ELEM_INSTR_RANGE)
Exception packets need their own ISA so the frontend can generate samples directly from them. Currently, ISA initialization is limited to instruction range packets.
Move ISA initialization into cs_etm_decoder__buffer_packet() and use a common helper to obtain it from the OpenCSD element. Populate the ISA for instruction ranges, context updates and exceptions, leaving it unknown for other element types.
Assisted-by: Codex:gpt-6 Signed-off-by: Leo Yan leo.yan@arm.com --- tools/perf/util/cs-etm-decoder/cs-etm-decoder.c | 69 +++++++++++++++---------- 1 file changed, 41 insertions(+), 28 deletions(-)
diff --git a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c index 35ca1a23e12735416d0d412a949595efc5bfe50a..469aba5e596a791289c5bcece132aba7b0baf6c3 100644 --- a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c +++ b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c @@ -368,9 +368,38 @@ cs_etm_decoder__reset_timestamp(struct cs_etm_packet_queue *packet_queue) packet_queue->instr_count = 0; }
+static enum cs_etm_isa +cs_etm_decoder__get_isa(const ocsd_generic_trace_elem *elem) +{ + /* + * OpenCSD preserves ISA and context between output elements, including + * exceptions after a context change with no intervening range. + */ + if (elem->elem_type != OCSD_GEN_TRC_ELEM_PE_CONTEXT && + elem->elem_type != OCSD_GEN_TRC_ELEM_INSTR_RANGE && + elem->elem_type != OCSD_GEN_TRC_ELEM_EXCEPTION) + return CS_ETM_ISA_UNKNOWN; + + switch (elem->isa) { + case ocsd_isa_aarch64: + return CS_ETM_ISA_A64; + case ocsd_isa_arm: + return CS_ETM_ISA_A32; + case ocsd_isa_thumb2: + return CS_ETM_ISA_T32; + case ocsd_isa_tee: + case ocsd_isa_jazelle: + case ocsd_isa_custom: + case ocsd_isa_unknown: + default: + return CS_ETM_ISA_UNKNOWN; + } +} + static ocsd_datapath_resp_t cs_etm_decoder__buffer_packet(struct cs_etm_queue *etmq, struct cs_etm_packet_queue *packet_queue, + const ocsd_generic_trace_elem *elem, const u8 trace_chan_id, enum cs_etm_sample_type sample_type) { @@ -389,7 +418,7 @@ cs_etm_decoder__buffer_packet(struct cs_etm_queue *etmq, packet_queue->packet_count++;
packet_queue->packet_buffer[et].sample_type = sample_type; - packet_queue->packet_buffer[et].isa = CS_ETM_ISA_UNKNOWN; + packet_queue->packet_buffer[et].isa = cs_etm_decoder__get_isa(elem); packet_queue->packet_buffer[et].cpu = cpu; packet_queue->packet_buffer[et].start_addr = CS_ETM_INVAL_ADDR; packet_queue->packet_buffer[et].end_addr = CS_ETM_INVAL_ADDR; @@ -420,31 +449,13 @@ cs_etm_decoder__buffer_range(struct cs_etm_queue *etmq, int ret = 0; struct cs_etm_packet *packet;
- ret = cs_etm_decoder__buffer_packet(etmq, packet_queue, trace_chan_id, - CS_ETM_RANGE); + ret = cs_etm_decoder__buffer_packet(etmq, packet_queue, elem, + trace_chan_id, CS_ETM_RANGE); if (ret != OCSD_RESP_CONT && ret != OCSD_RESP_WAIT) return ret;
packet = &packet_queue->packet_buffer[packet_queue->tail];
- switch (elem->isa) { - case ocsd_isa_aarch64: - packet->isa = CS_ETM_ISA_A64; - break; - case ocsd_isa_arm: - packet->isa = CS_ETM_ISA_A32; - break; - case ocsd_isa_thumb2: - packet->isa = CS_ETM_ISA_T32; - break; - case ocsd_isa_tee: - case ocsd_isa_jazelle: - case ocsd_isa_custom: - case ocsd_isa_unknown: - default: - packet->isa = CS_ETM_ISA_UNKNOWN; - } - packet->start_addr = elem->st_addr; packet->end_addr = elem->en_addr; packet->instr_count = elem->num_instr_range; @@ -483,6 +494,7 @@ cs_etm_decoder__buffer_range(struct cs_etm_queue *etmq, static ocsd_datapath_resp_t cs_etm_decoder__buffer_discontinuity(struct cs_etm_queue *etmq, struct cs_etm_packet_queue *queue, + const ocsd_generic_trace_elem *elem, const uint8_t trace_chan_id) { /* @@ -490,7 +502,7 @@ cs_etm_decoder__buffer_discontinuity(struct cs_etm_queue *etmq, * reset time statistics. */ cs_etm_decoder__reset_timestamp(queue); - return cs_etm_decoder__buffer_packet(etmq, queue, trace_chan_id, + return cs_etm_decoder__buffer_packet(etmq, queue, elem, trace_chan_id, CS_ETM_DISCONTINUITY); }
@@ -502,7 +514,7 @@ cs_etm_decoder__buffer_exception(struct cs_etm_queue *etmq, { int ret = 0; struct cs_etm_packet *packet;
- ret = cs_etm_decoder__buffer_packet(etmq, queue, trace_chan_id, + ret = cs_etm_decoder__buffer_packet(etmq, queue, elem, trace_chan_id, CS_ETM_EXCEPTION); if (ret != OCSD_RESP_CONT && ret != OCSD_RESP_WAIT) return ret; @@ -516,9 +528,10 @@ cs_etm_decoder__buffer_exception(struct cs_etm_queue *etmq, static ocsd_datapath_resp_t cs_etm_decoder__buffer_exception_ret(struct cs_etm_queue *etmq, struct cs_etm_packet_queue *queue, + const ocsd_generic_trace_elem *elem, const uint8_t trace_chan_id) { - return cs_etm_decoder__buffer_packet(etmq, queue, trace_chan_id, + return cs_etm_decoder__buffer_packet(etmq, queue, elem, trace_chan_id, CS_ETM_EXCEPTION_RET); }
@@ -555,8 +568,8 @@ cs_etm_decoder__set_tid(struct cs_etm_queue *etmq, elem->context.exception_level, tid)) return OCSD_RESP_FATAL_SYS_ERR;
- ret = cs_etm_decoder__buffer_packet(etmq, packet_queue, trace_chan_id, - CS_ETM_CONTEXT); + ret = cs_etm_decoder__buffer_packet(etmq, packet_queue, elem, + trace_chan_id, CS_ETM_CONTEXT); if (ret != OCSD_RESP_CONT && ret != OCSD_RESP_WAIT) return ret;
@@ -597,7 +610,7 @@ static ocsd_datapath_resp_t cs_etm_decoder__gen_trace_elem_printer( type == OCSD_GEN_TRC_ELEM_TRACE_ON || type == OCSD_GEN_TRC_ELEM_ADDR_NACC) resp = cs_etm_decoder__buffer_discontinuity(etmq, packet_queue, - trace_chan_id); + elem, trace_chan_id); else if (type == OCSD_GEN_TRC_ELEM_INSTR_RANGE) resp = cs_etm_decoder__buffer_range(etmq, packet_queue, elem, trace_chan_id); @@ -606,7 +619,7 @@ static ocsd_datapath_resp_t cs_etm_decoder__gen_trace_elem_printer( trace_chan_id); else if (type == OCSD_GEN_TRC_ELEM_EXCEPTION_RET) resp = cs_etm_decoder__buffer_exception_ret(etmq, packet_queue, - trace_chan_id); + elem, trace_chan_id); else if (type == OCSD_GEN_TRC_ELEM_TIMESTAMP) resp = cs_etm_decoder__do_hard_timestamp(etmq, elem, trace_chan_id,
Use packet->last_instr_size in cs_etm__instr_size() instead of a hard-coded value, allowing the decoder to supply the instruction size.
Keep per-address decoding for T32. A range can mix 16-bit and 32-bit instructions, so last_instr_size describes only the final instruction and cannot determine the size at an arbitrary address in the range.
Assisted-by: Codex:gpt-6 Signed-off-by: Leo Yan leo.yan@arm.com --- tools/perf/util/cs-etm.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/tools/perf/util/cs-etm.c b/tools/perf/util/cs-etm.c index 2d1ab34f7b6b5a6331005c16c6a418d123c815eb..0163d93ac2d537a2190536e77d24628b4d707751 100644 --- a/tools/perf/util/cs-etm.c +++ b/tools/perf/util/cs-etm.c @@ -1358,8 +1358,7 @@ static inline int cs_etm__instr_size(struct cs_etm_queue *etmq, if (packet->isa == CS_ETM_ISA_T32) return cs_etm__t32_instr_size(etmq, tidq, packet, addr);
- /* Otherwise, 4-byte instruction size for A32/A64 */ - return 4; + return packet->last_instr_size; }
static inline u64 cs_etm__first_executed_instr(struct cs_etm_packet *packet)
On 23/09/2026 16:21, Leo Yan wrote:
Use packet->last_instr_size in cs_etm__instr_size() instead of a hard-coded value, allowing the decoder to supply the instruction size.
Keep per-address decoding for T32. A range can mix 16-bit and 32-bit instructions, so last_instr_size describes only the final instruction and cannot determine the size at an arbitrary address in the range.
Assisted-by: Codex:gpt-6 Signed-off-by: Leo Yan leo.yan@arm.com
tools/perf/util/cs-etm.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/tools/perf/util/cs-etm.c b/tools/perf/util/cs-etm.c index 2d1ab34f7b6b5a6331005c16c6a418d123c815eb..0163d93ac2d537a2190536e77d24628b4d707751 100644 --- a/tools/perf/util/cs-etm.c +++ b/tools/perf/util/cs-etm.c @@ -1358,8 +1358,7 @@ static inline int cs_etm__instr_size(struct cs_etm_queue *etmq, if (packet->isa == CS_ETM_ISA_T32) return cs_etm__t32_instr_size(etmq, tidq, packet, addr);
- /* Otherwise, 4-byte instruction size for A32/A64 */
- return 4;
- return packet->last_instr_size;
Why not use last_instr_size for CS_ETM_ISA_T32 too? I'm wondering how and when this change actually makes a difference? last_instr_size isn't any better than '4' for any instruction in the range other than the last one, so it's not clear what the issue is.
} static inline u64 cs_etm__first_executed_instr(struct cs_etm_packet *packet)
An exception can transfer execution to the kernel after any instruction. When this creates a trace boundary, perf may attribute the trace-end sample to the last instruction in the range. If that instruction is an untaken branch, reporting only its branch type can make the transfer look like a taken branch.
Set PERF_IP_FLAG_NOT_TAKEN for branch ranges when last_instr_taken_branch is false. This records the decoded branch outcome independently of exception entry.
For a discontinuity without an exception element, perf script previously reported an untaken B.LS as:
tr end jcc 4000f8 => 0 b.ls #0x400118 tr strt jmp 0 => 4000fc
With the not-taken flag set:
tr end jcc/not_taken/ 4000f8 => 0 b.ls #0x400118 tr strt jmp 0 => 4000fc
Here, 4000f8 remains the last traced instruction and 4000fc is the trace restart address. These boundary samples do not establish an IRQ entry or an exception return; the not_taken flag records only the branch outcome.
Assisted-by: Codex:gpt-6 Signed-off-by: Leo Yan leo.yan@arm.com --- tools/perf/util/cs-etm.c | 5 +++++ 1 file changed, 5 insertions(+)
diff --git a/tools/perf/util/cs-etm.c b/tools/perf/util/cs-etm.c index 0163d93ac2d537a2190536e77d24628b4d707751..c54218aeef53e8420447e9a2bee6e98d8225e335 100644 --- a/tools/perf/util/cs-etm.c +++ b/tools/perf/util/cs-etm.c @@ -2415,6 +2415,11 @@ static int cs_etm__set_sample_flags(struct cs_etm_queue *etmq, packet->flags = PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_RETURN;
+ /* Report branch that is not taken */ + if ((packet->flags & PERF_IP_FLAG_BRANCH) && + !packet->last_instr_taken_branch) + packet->flags |= PERF_IP_FLAG_NOT_TAKEN; + /* * Decoder might insert a discontinuity in the middle of * instruction packets, fixup prev_packet with flag
cs_etm__flush() and cs_etm__end_block() duplicate the instruction sample used to report the remaining branch history at the end of a trace.
Move this code into cs_etm__synth_last_instruction_sample(), preserving the sample generation and instruction period reset.
Assisted-by: Codex:gpt-6 Signed-off-by: Leo Yan leo.yan@arm.com --- tools/perf/util/cs-etm.c | 71 ++++++++++++++++++------------------------------ 1 file changed, 26 insertions(+), 45 deletions(-)
diff --git a/tools/perf/util/cs-etm.c b/tools/perf/util/cs-etm.c index c54218aeef53e8420447e9a2bee6e98d8225e335..5b5ee52c0277f5ff8f03c2908e4af1d7801845ff 100644 --- a/tools/perf/util/cs-etm.c +++ b/tools/perf/util/cs-etm.c @@ -1685,6 +1685,28 @@ static int cs_etm__synth_instruction_sample(struct cs_etm_queue *etmq, return ret; }
+static int cs_etm__synth_last_instruction_sample(struct cs_etm_queue *etmq, + struct cs_etm_traceid_queue *tidq) +{ + struct cs_etm_packet *packet = tidq->prev_packet; + int ret; + + if (!etmq->etm->synth_opts.last_branch || + !etmq->etm->synth_opts.instructions) + return 0; + + if (packet->sample_type != CS_ETM_RANGE) + return 0; + + ret = cs_etm__synth_instruction_sample(etmq, tidq, packet, + cs_etm__last_executed_instr(packet), + tidq->period_instructions); + if (!ret) + tidq->period_instructions = 0; + + return ret; +} + /* * The cs etm packet encodes an instruction range between a branch target * and the next taken branch. Generate sample accordingly. @@ -2023,28 +2045,9 @@ static int cs_etm__flush(struct cs_etm_queue *etmq, if (tidq->prev_packet->sample_type == CS_ETM_EMPTY) goto swap_packet;
- if (etmq->etm->synth_opts.last_branch && - etmq->etm->synth_opts.instructions && - tidq->prev_packet->sample_type == CS_ETM_RANGE) { - u64 addr; - /* - * Generate a last branch event for the branches left in the - * circular buffer at the end of the trace. - * - * Use the address of the end of the last reported execution - * range - */ - addr = cs_etm__last_executed_instr(tidq->prev_packet); - - err = cs_etm__synth_instruction_sample( - etmq, tidq, tidq->prev_packet, addr, - tidq->period_instructions); - if (err) - return err; - - tidq->period_instructions = 0; - - } + err = cs_etm__synth_last_instruction_sample(etmq, tidq); + if (err) + return err;
if (etm->synth_opts.branches && tidq->prev_packet->sample_type == CS_ETM_RANGE) { @@ -2066,8 +2069,6 @@ static int cs_etm__flush(struct cs_etm_queue *etmq, static int cs_etm__end_block(struct cs_etm_queue *etmq, struct cs_etm_traceid_queue *tidq) { - int err; - /* * It has no new packet coming and 'etmq->packet' contains the stale * packet which was set at the previous time with packets swapping; @@ -2077,27 +2078,7 @@ static int cs_etm__end_block(struct cs_etm_queue *etmq, * event for the branches left in the circular buffer at the end of * the trace. */ - if (etmq->etm->synth_opts.last_branch && - etmq->etm->synth_opts.instructions && - tidq->prev_packet->sample_type == CS_ETM_RANGE) { - u64 addr; - - /* - * Use the address of the end of the last reported execution - * range. - */ - addr = cs_etm__last_executed_instr(tidq->prev_packet); - - err = cs_etm__synth_instruction_sample( - etmq, tidq, tidq->prev_packet, addr, - tidq->period_instructions); - if (err) - return err; - - tidq->period_instructions = 0; - } - - return 0; + return cs_etm__synth_last_instruction_sample(etmq, tidq); }
static int cs_etm__flush_stack_cb(struct thread *thread,
Move the branch-synthesis option and packet checks from cs_etm__sample() into cs_etm__synth_branch_sample(), alongside the existing branch filter. Call the helper directly from cs_etm__sample() and cs_etm__flush().
Accept trace-start and trace-end flags, this preserves boundary samples without a taken branch.
Assisted-by: Codex:gpt-6 Signed-off-by: Leo Yan leo.yan@arm.com --- tools/perf/util/cs-etm.c | 37 ++++++++++++++----------------------- 1 file changed, 14 insertions(+), 23 deletions(-)
diff --git a/tools/perf/util/cs-etm.c b/tools/perf/util/cs-etm.c index 5b5ee52c0277f5ff8f03c2908e4af1d7801845ff..c8a92b5dba6001329d68766d540b9b7ead90e9e6 100644 --- a/tools/perf/util/cs-etm.c +++ b/tools/perf/util/cs-etm.c @@ -1726,6 +1726,14 @@ static int cs_etm__synth_branch_sample(struct cs_etm_queue *etmq, } dummy_bs; u64 ip;
+ if (!etm->synth_opts.branches) + return 0; + + if (!cs_etm__packet_has_taken_branch(tidq->prev_packet) && + !(tidq->prev_packet->flags & (PERF_IP_FLAG_TRACE_BEGIN | + PERF_IP_FLAG_TRACE_END))) + return 0; + if (etm->branches_filter && !(etm->branches_filter & tidq->prev_packet->flags)) return 0; @@ -1965,23 +1973,9 @@ static int cs_etm__sample(struct cs_etm_queue *etmq, } }
- if (etm->synth_opts.branches) { - bool generate_sample = false; - - /* Generate sample for tracing on packet */ - if (tidq->prev_packet->sample_type == CS_ETM_DISCONTINUITY) - generate_sample = true; - - /* Generate sample for branch taken packet */ - if (cs_etm__packet_has_taken_branch(tidq->prev_packet)) - generate_sample = true; - - if (generate_sample) { - ret = cs_etm__synth_branch_sample(etmq, tidq); - if (ret) - return ret; - } - } + ret = cs_etm__synth_branch_sample(etmq, tidq); + if (ret) + return ret;
cs_etm__packet_swap(etm, tidq);
@@ -2049,12 +2043,9 @@ static int cs_etm__flush(struct cs_etm_queue *etmq, if (err) return err;
- if (etm->synth_opts.branches && - tidq->prev_packet->sample_type == CS_ETM_RANGE) { - err = cs_etm__synth_branch_sample(etmq, tidq); - if (err) - return err; - } + err = cs_etm__synth_branch_sample(etmq, tidq); + if (err) + return err;
swap_packet: cs_etm__packet_swap(etm, tidq);
ETMv4 and ETE use the same exception number for SVC, HVC and SMC. Identify SVC by inspecting the instruction before the preferred return address in the exception packet.
Preserve the exception level and preferred return address from OpenCSD, and use them with the packet's ISA in cs_etm__is_syscall() and cs_etm__is_sync_exception(). This makes exception call classification independent of the preceding instruction range.
Assisted-by: Codex:gpt-6 Signed-off-by: Leo Yan leo.yan@arm.com --- tools/perf/util/cs-etm-decoder/cs-etm-decoder.c | 7 ++++++- tools/perf/util/cs-etm.c | 10 +++------- tools/perf/util/cs-etm.h | 1 + 3 files changed, 10 insertions(+), 8 deletions(-)
diff --git a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c index 469aba5e596a791289c5bcece132aba7b0baf6c3..48e8c355a3e495cc30dcb4056c04a034d10ce860 100644 --- a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c +++ b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c @@ -511,7 +511,8 @@ cs_etm_decoder__buffer_exception(struct cs_etm_queue *etmq, struct cs_etm_packet_queue *queue, const ocsd_generic_trace_elem *elem, const uint8_t trace_chan_id) -{ int ret = 0; +{ + int ret = 0; struct cs_etm_packet *packet;
ret = cs_etm_decoder__buffer_packet(etmq, queue, elem, trace_chan_id, @@ -521,6 +522,10 @@ cs_etm_decoder__buffer_exception(struct cs_etm_queue *etmq,
packet = &queue->packet_buffer[queue->tail]; packet->exception_number = elem->exception_number; + if (elem->context.el_valid) + packet->el = elem->context.exception_level; + if (elem->excep_ret_addr) + packet->end_addr = elem->en_addr;
return ret; } diff --git a/tools/perf/util/cs-etm.c b/tools/perf/util/cs-etm.c index c8a92b5dba6001329d68766d540b9b7ead90e9e6..a40b3b2fe2e6318154111d9b7ebd05bee53934f1 100644 --- a/tools/perf/util/cs-etm.c +++ b/tools/perf/util/cs-etm.c @@ -2019,7 +2019,7 @@ static int cs_etm__exception(struct cs_etm_traceid_queue *tidq) * to generate branch sample for the instruction range before the * exception is trapped to kernel or before the exception returning. * - * The exception packet includes the dummy address values, so don't + * The exception packet does not describe an instruction range, so don't * swap PACKET with PREV_PACKET. This keeps PREV_PACKET to be useful * for generating instruction and branch samples. */ @@ -2225,7 +2225,6 @@ static bool cs_etm__is_syscall(struct cs_etm_queue *etmq, struct cs_etm_traceid_queue *tidq, u64 magic) { struct cs_etm_packet *packet = tidq->packet; - struct cs_etm_packet *prev_packet = tidq->prev_packet;
if (magic == __perf_cs_etmv3_magic) if (packet->exception_number == CS_ETMV3_EXC_SVC) @@ -2238,8 +2237,7 @@ static bool cs_etm__is_syscall(struct cs_etm_queue *etmq, */ if (magic == __perf_cs_etmv4_magic || magic == __perf_cs_ete_magic) { if (packet->exception_number == CS_ETMV4_EXC_CALL && - cs_etm__is_svc_instr(etmq, tidq, prev_packet, - prev_packet->end_addr)) + cs_etm__is_svc_instr(etmq, tidq, packet, packet->end_addr)) return true; }
@@ -2277,7 +2275,6 @@ static bool cs_etm__is_sync_exception(struct cs_etm_queue *etmq, u64 magic) { struct cs_etm_packet *packet = tidq->packet; - struct cs_etm_packet *prev_packet = tidq->prev_packet;
if (magic == __perf_cs_etmv3_magic) if (packet->exception_number == CS_ETMV3_EXC_SMC || @@ -2301,8 +2298,7 @@ static bool cs_etm__is_sync_exception(struct cs_etm_queue *etmq, * (SMC, HVC) are taken as sync exceptions. */ if (packet->exception_number == CS_ETMV4_EXC_CALL && - !cs_etm__is_svc_instr(etmq, tidq, prev_packet, - prev_packet->end_addr)) + !cs_etm__is_svc_instr(etmq, tidq, packet, packet->end_addr)) return true;
/* diff --git a/tools/perf/util/cs-etm.h b/tools/perf/util/cs-etm.h index b81099c2b301c25f894cc9a50edfde87aceff49d..4d03f2a680b58aacd91c9f6c957fb2d38e7ac2ad 100644 --- a/tools/perf/util/cs-etm.h +++ b/tools/perf/util/cs-etm.h @@ -174,6 +174,7 @@ struct cs_etm_packet { enum cs_etm_sample_type sample_type; enum cs_etm_isa isa; u64 start_addr; + /* For exceptions: preferred return address, or CS_ETM_INVAL_ADDR. */ u64 end_addr; u32 instr_count; u32 last_instr_type;
On 23/09/2026 16:21, Leo Yan wrote:
ETMv4 and ETE use the same exception number for SVC, HVC and SMC. Identify SVC by inspecting the instruction before the preferred return address in the exception packet.
Preserve the exception level and preferred return address from OpenCSD, and use them with the packet's ISA in cs_etm__is_syscall() and cs_etm__is_sync_exception(). This makes exception call classification independent of the preceding instruction range.
Assisted-by: Codex:gpt-6 Signed-off-by: Leo Yan leo.yan@arm.com
tools/perf/util/cs-etm-decoder/cs-etm-decoder.c | 7 ++++++- tools/perf/util/cs-etm.c | 10 +++------- tools/perf/util/cs-etm.h | 1 + 3 files changed, 10 insertions(+), 8 deletions(-)
diff --git a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c index 469aba5e596a791289c5bcece132aba7b0baf6c3..48e8c355a3e495cc30dcb4056c04a034d10ce860 100644 --- a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c +++ b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c @@ -511,7 +511,8 @@ cs_etm_decoder__buffer_exception(struct cs_etm_queue *etmq, struct cs_etm_packet_queue *queue, const ocsd_generic_trace_elem *elem, const uint8_t trace_chan_id) -{ int ret = 0; +{
- int ret = 0; struct cs_etm_packet *packet;
ret = cs_etm_decoder__buffer_packet(etmq, queue, elem, trace_chan_id, @@ -521,6 +522,10 @@ cs_etm_decoder__buffer_exception(struct cs_etm_queue *etmq, packet = &queue->packet_buffer[queue->tail]; packet->exception_number = elem->exception_number;
- if (elem->context.el_valid)
packet->el = elem->context.exception_level;- if (elem->excep_ret_addr)
packet->end_addr = elem->en_addr;return ret; } diff --git a/tools/perf/util/cs-etm.c b/tools/perf/util/cs-etm.c index c8a92b5dba6001329d68766d540b9b7ead90e9e6..a40b3b2fe2e6318154111d9b7ebd05bee53934f1 100644 --- a/tools/perf/util/cs-etm.c +++ b/tools/perf/util/cs-etm.c @@ -2019,7 +2019,7 @@ static int cs_etm__exception(struct cs_etm_traceid_queue *tidq) * to generate branch sample for the instruction range before the * exception is trapped to kernel or before the exception returning. *
* The exception packet includes the dummy address values, so don't
* The exception packet does not describe an instruction range, so don't*/
- swap PACKET with PREV_PACKET. This keeps PREV_PACKET to be useful
- for generating instruction and branch samples.
@@ -2225,7 +2225,6 @@ static bool cs_etm__is_syscall(struct cs_etm_queue *etmq, struct cs_etm_traceid_queue *tidq, u64 magic) { struct cs_etm_packet *packet = tidq->packet;
- struct cs_etm_packet *prev_packet = tidq->prev_packet;
if (magic == __perf_cs_etmv3_magic) if (packet->exception_number == CS_ETMV3_EXC_SVC) @@ -2238,8 +2237,7 @@ static bool cs_etm__is_syscall(struct cs_etm_queue *etmq, */ if (magic == __perf_cs_etmv4_magic || magic == __perf_cs_ete_magic) { if (packet->exception_number == CS_ETMV4_EXC_CALL &&
cs_etm__is_svc_instr(etmq, tidq, prev_packet,prev_packet->end_addr))
cs_etm__is_svc_instr(etmq, tidq, packet, packet->end_addr)) return true;@@ -2277,7 +2275,6 @@ static bool cs_etm__is_sync_exception(struct cs_etm_queue *etmq, u64 magic) { struct cs_etm_packet *packet = tidq->packet;
- struct cs_etm_packet *prev_packet = tidq->prev_packet;
if (magic == __perf_cs_etmv3_magic) if (packet->exception_number == CS_ETMV3_EXC_SMC || @@ -2301,8 +2298,7 @@ static bool cs_etm__is_sync_exception(struct cs_etm_queue *etmq, * (SMC, HVC) are taken as sync exceptions. */ if (packet->exception_number == CS_ETMV4_EXC_CALL &&
!cs_etm__is_svc_instr(etmq, tidq, prev_packet,prev_packet->end_addr))
!cs_etm__is_svc_instr(etmq, tidq, packet, packet->end_addr)) return true;/* diff --git a/tools/perf/util/cs-etm.h b/tools/perf/util/cs-etm.h index b81099c2b301c25f894cc9a50edfde87aceff49d..4d03f2a680b58aacd91c9f6c957fb2d38e7ac2ad 100644 --- a/tools/perf/util/cs-etm.h +++ b/tools/perf/util/cs-etm.h @@ -174,6 +174,7 @@ struct cs_etm_packet { enum cs_etm_sample_type sample_type; enum cs_etm_isa isa; u64 start_addr;
- /* For exceptions: preferred return address, or CS_ETM_INVAL_ADDR. */ u64 end_addr;
Minor nit, but you could use a union. I don't know how clear re-using "end_addr" id for exception packets is when reading the code.
union { u64 end_addr; // Range packets u64 pref_ret_addr; // Exception packets };
u32 instr_count; u32 last_instr_type;
Exception entry is currently represented by replacing the preceding range's flags and forcing its last_instr_taken_branch bit. This gives an interrupt the source address of the last executed instruction. When that instruction is a taken branch, its edge and original flags are lost.
For example, consider an untaken B.LS followed by an IRQ:
4000f4: eb02003f cmp x1, x2 4000f8: 54000109 b.ls 400118 4000fc: d282f2c2 mov x2, #0x1796
If B.LS completes untaken and the IRQ is taken before MOV completes, the preferred exception return address is 4000fc. With kernel tracing disabled, perf script previously attributed the IRQ to the preceding range:
tr end hw int 4000f8 => 0 b.ls #0x400118 tr strt jmp 0 => 4000fc
When an IRQ element provides that return address, use it as the source of the synthesized exception sample:
tr end hw int 4000fc => 0 movz x2, #0x1796 tr strt jmp 0 => 4000fc
The source now identifies the architectural resume PC, rather than the last completed instruction. The IRQ signal could have arrived while B.LS was executing. The traced preferred exception return address of 4000fc confirms that B.LS had retired architecturally before the IRQ was taken.
OpenCSD provides an exception's preferred return address in en_addr when excep_ret_addr is set. It does not define or initialize st_addr for an exception element. Derive A32 and A64 exception source PCs from the preferred return address. Exception calls (SVC/SMC/HVC) use the preceding four-byte instruction; interrupts, faults and traps use the return address with last_instr_size set to zero.
Keep T32 exception samples on the existing fallback path, so determining the call size requires an instruction read. Retain end_addr for the frontend's SVC check. RESET and missing return addresses also use the fallback because their source PC is unknown.
Resolve the preceding branch with its original flags without adding instructions. Then retain the exception in prev_packet until the next range, exception or discontinuity resolves its destination. This preserves both edges when an exception follows a taken branch and also handles consecutive exceptions.
Supply the preferred return address in sample.ret_addr so later instruction fetching cannot change the return PC used by call/return export. Generate final instruction samples only from nonempty instruction ranges.
CS_ETM_EXCEPTION_RET needs no further handling in the packet processing loop because ERET is already part of the preceding instruction range. cs_etm__set_sample_flags() now sets both that range's return flags and last_instr_taken_branch.
Assisted-by: Codex:gpt-6 Signed-off-by: Leo Yan leo.yan@arm.com --- tools/perf/util/cs-etm-decoder/cs-etm-decoder.c | 31 ++++++- tools/perf/util/cs-etm.c | 102 ++++++++++++++---------- tools/perf/util/cs-etm.h | 1 + 3 files changed, 91 insertions(+), 43 deletions(-)
diff --git a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c index 48e8c355a3e495cc30dcb4056c04a034d10ce860..83a851bdd87bf0aa3fbc565305451a68f42c69e6 100644 --- a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c +++ b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c @@ -514,6 +514,7 @@ cs_etm_decoder__buffer_exception(struct cs_etm_queue *etmq, { int ret = 0; struct cs_etm_packet *packet; + u8 last_instr_size = 0;
ret = cs_etm_decoder__buffer_packet(etmq, queue, elem, trace_chan_id, CS_ETM_EXCEPTION); @@ -524,8 +525,34 @@ cs_etm_decoder__buffer_exception(struct cs_etm_queue *etmq, packet->exception_number = elem->exception_number; if (elem->context.el_valid) packet->el = elem->context.exception_level; - if (elem->excep_ret_addr) - packet->end_addr = elem->en_addr; + + /* RESET has no defined preferred return address. */ + if (!elem->excep_ret_addr || + packet->exception_number == CS_ETMV4_EXC_RESET) + return ret; + + packet->end_addr = elem->en_addr; + + /* + * T32 calls can be two or four bytes, requiring an instruction read + * before end_addr to determine their size. Keep end_addr for the + * frontend's SVC check and leave start_addr unknown for its fallback. + */ + if (packet->isa != CS_ETM_ISA_A32 && packet->isa != CS_ETM_ISA_A64) + return ret; + + /* + * For A32 and A64, exception calls execute a four-byte SVC, HVC or + * SMC and save the following PC. Other exceptions use the interrupted + * or faulting PC, including traps on those instructions. + * + * With a shared branch target, no instruction has completed + * at the target, so last_instr_size remains zero. + */ + if (elem->exception_number == CS_ETMV4_EXC_CALL) + last_instr_size = 4; + packet->start_addr = packet->end_addr - last_instr_size; + packet->last_instr_size = last_instr_size;
return ret; } diff --git a/tools/perf/util/cs-etm.c b/tools/perf/util/cs-etm.c index a40b3b2fe2e6318154111d9b7ebd05bee53934f1..6641e257e5f18f30a185a88db720c1404cf2bd36 100644 --- a/tools/perf/util/cs-etm.c +++ b/tools/perf/util/cs-etm.c @@ -101,10 +101,8 @@ struct cs_etm_traceid_queue { ocsd_ex_level decode_el;
/* - * The frontend accesses the EL from '[prev_]packet' because it needs - * previous EL for branch and current EL for instruction samples. It's - * not possible to change thread in a single branch sample so no need to - * store or access the thread through the packet. + * Samples use the EL saved in their source packet. A branch sample + * cannot change thread, so the thread is kept in the frontend context. */ struct thread *frontend_thread; }; @@ -1364,11 +1362,10 @@ static inline int cs_etm__instr_size(struct cs_etm_queue *etmq, static inline u64 cs_etm__first_executed_instr(struct cs_etm_packet *packet) { /* - * Return 0 for packets that have no addresses so that CS_ETM_INVAL_ADDR doesn't - * appear in samples. + * Return 0 for discontinuities so that CS_ETM_INVAL_ADDR doesn't appear + * in samples. */ - if (packet->sample_type == CS_ETM_DISCONTINUITY || - packet->sample_type == CS_ETM_EXCEPTION) + if (packet->sample_type == CS_ETM_DISCONTINUITY) return 0;
return packet->start_addr; @@ -1537,6 +1534,8 @@ static void cs_etm__copy_insn(struct cs_etm_queue *etmq, }
sample->insn_len = cs_etm__instr_size(etmq, tidq, packet, sample->ip); + if (packet->sample_type == CS_ETM_EXCEPTION && !sample->insn_len) + return;
cs_etm__frontend_mem_access(etmq, tidq, packet, sample->ip, sample->insn_len, (void *)sample->insn); @@ -1564,8 +1563,11 @@ static inline u64 cs_etm__resolve_sample_time(struct cs_etm_queue *etmq, return etm->latest_kernel_timestamp; }
-static bool cs_etm__packet_has_taken_branch(struct cs_etm_packet *packet) +static bool cs_etm__packet_has_branch(struct cs_etm_packet *packet) { + if (packet->sample_type == CS_ETM_EXCEPTION) + return true; + if (packet->sample_type == CS_ETM_RANGE && packet->last_instr_taken_branch) return true; @@ -1583,7 +1585,7 @@ static void cs_etm__add_stack_event(struct cs_etm_queue *etmq, if (!etm->synth_opts.branches && !etm->synth_opts.instructions) return;
- if (!cs_etm__packet_has_taken_branch(tidq->prev_packet)) + if (!cs_etm__packet_has_branch(tidq->prev_packet)) return;
if (etmq->etm->use_thread_stack) { @@ -1695,7 +1697,8 @@ static int cs_etm__synth_last_instruction_sample(struct cs_etm_queue *etmq, !etmq->etm->synth_opts.instructions) return 0;
- if (packet->sample_type != CS_ETM_RANGE) + /* Only nonempty ranges provide a final instruction to sample. */ + if (packet->sample_type != CS_ETM_RANGE || !packet->instr_count) return 0;
ret = cs_etm__synth_instruction_sample(etmq, tidq, packet, @@ -1729,7 +1732,7 @@ static int cs_etm__synth_branch_sample(struct cs_etm_queue *etmq, if (!etm->synth_opts.branches) return 0;
- if (!cs_etm__packet_has_taken_branch(tidq->prev_packet) && + if (!cs_etm__packet_has_branch(tidq->prev_packet) && !(tidq->prev_packet->flags & (PERF_IP_FLAG_TRACE_BEGIN | PERF_IP_FLAG_TRACE_END))) return 0; @@ -1760,6 +1763,9 @@ static int cs_etm__synth_branch_sample(struct cs_etm_queue *etmq, sample.flags = tidq->prev_packet->flags; sample.cpumode = event->sample.header.misc;
+ if (tidq->prev_packet->sample_type == CS_ETM_EXCEPTION) + sample.ret_addr = tidq->prev_packet->end_addr; + cs_etm__copy_insn(etmq, tidq, tidq->prev_packet, &sample);
/* @@ -2010,21 +2016,23 @@ static int cs_etm__context(struct cs_etm_queue *etmq, return ret; }
-static int cs_etm__exception(struct cs_etm_traceid_queue *tidq) +static int cs_etm__exception(struct cs_etm_queue *etmq, + struct cs_etm_traceid_queue *tidq) { + struct cs_etm_packet *packet = tidq->packet; + /* - * When the exception packet is inserted, whether the last instruction - * in previous range packet is taken branch or not, we need to force - * to set 'prev_packet->last_instr_taken_branch' to true. This ensures - * to generate branch sample for the instruction range before the - * exception is trapped to kernel or before the exception returning. - * - * The exception packet does not describe an instruction range, so don't - * swap PACKET with PREV_PACKET. This keeps PREV_PACKET to be useful - * for generating instruction and branch samples. + * Resolve the preceding branch without adding instructions, then keep + * this exception as prev_packet until its destination is known. */ - if (tidq->prev_packet->sample_type == CS_ETM_RANGE) + if (packet->start_addr != CS_ETM_INVAL_ADDR) + return cs_etm__sample(etmq, tidq); + + /* Fall back to attributing the exception to the preceding range. */ + if (tidq->prev_packet->sample_type == CS_ETM_RANGE) { + tidq->prev_packet->flags = packet->flags; tidq->prev_packet->last_instr_taken_branch = true; + }
return 0; } @@ -2418,7 +2426,8 @@ static int cs_etm__set_sample_flags(struct cs_etm_queue *etmq, * instruction packet, set flag PERF_IP_FLAG_TRACE_END * for previous packet. */ - if (prev_packet->sample_type == CS_ETM_RANGE) + if (prev_packet->sample_type == CS_ETM_RANGE || + prev_packet->sample_type == CS_ETM_EXCEPTION) prev_packet->flags |= PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_TRACE_END; break; @@ -2450,15 +2459,23 @@ static int cs_etm__set_sample_flags(struct cs_etm_queue *etmq, PERF_IP_FLAG_CALL | PERF_IP_FLAG_INTERRUPT;
- /* - * When the exception packet is inserted, since exception - * packet is not used standalone for generating samples - * and it's affiliation to the previous instruction range - * packet; so set previous range packet flags to tell perf - * it is an exception taken branch. - */ - if (prev_packet->sample_type == CS_ETM_RANGE) - prev_packet->flags = packet->flags; + if (packet->start_addr == CS_ETM_INVAL_ADDR) + break; + + /* Resolve the preceding trace start or exception return. */ + if (prev_packet->sample_type == CS_ETM_DISCONTINUITY) + prev_packet->flags |= PERF_IP_FLAG_BRANCH | + PERF_IP_FLAG_TRACE_BEGIN; + + if (prev_packet->flags == (PERF_IP_FLAG_BRANCH | + PERF_IP_FLAG_RETURN | + PERF_IP_FLAG_INTERRUPT) && + cs_etm__is_svc_instr(etmq, tidq, packet, + packet->start_addr)) { + prev_packet->flags = PERF_IP_FLAG_BRANCH | + PERF_IP_FLAG_RETURN | + PERF_IP_FLAG_SYSCALLRET; + } break; case CS_ETM_EXCEPTION_RET: /* @@ -2486,10 +2503,12 @@ static int cs_etm__set_sample_flags(struct cs_etm_queue *etmq, * system call instruction and then calibrate the sample flag * as needed. */ - if (prev_packet->sample_type == CS_ETM_RANGE) + if (prev_packet->sample_type == CS_ETM_RANGE) { prev_packet->flags = PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_RETURN | PERF_IP_FLAG_INTERRUPT; + prev_packet->last_instr_taken_branch = true; + } break; case CS_ETM_CONTEXT: case CS_ETM_EMPTY: @@ -2565,7 +2584,9 @@ static int cs_etm__process_traceid_queue(struct cs_etm_queue *etmq, * range, generate instruction sequence * events. */ - cs_etm__sample(etmq, tidq); + ret = cs_etm__sample(etmq, tidq); + if (ret) + goto out; break; case CS_ETM_CONTEXT: /* @@ -2579,13 +2600,12 @@ static int cs_etm__process_traceid_queue(struct cs_etm_queue *etmq, goto out; break; case CS_ETM_EXCEPTION: + ret = cs_etm__exception(etmq, tidq); + if (ret) + goto out; + break; case CS_ETM_EXCEPTION_RET: - /* - * If the exception packet is coming, - * make sure the previous instruction - * range packet to be handled properly. - */ - cs_etm__exception(tidq); + /* The return annotates the preceding instruction range. */ break; case CS_ETM_DISCONTINUITY: /* diff --git a/tools/perf/util/cs-etm.h b/tools/perf/util/cs-etm.h index 4d03f2a680b58aacd91c9f6c957fb2d38e7ac2ad..27201fea5af2764adf85a766925c393ad93d40f8 100644 --- a/tools/perf/util/cs-etm.h +++ b/tools/perf/util/cs-etm.h @@ -173,6 +173,7 @@ struct cs_etm_queue; struct cs_etm_packet { enum cs_etm_sample_type sample_type; enum cs_etm_isa isa; + /* For exceptions: source PC, or CS_ETM_INVAL_ADDR. */ u64 start_addr; /* For exceptions: preferred return address, or CS_ETM_INVAL_ADDR. */ u64 end_addr;
Add an AArch64 branch_not_taken_loop workload and a CoreSight shell test checking IRQ entry and exit PCs. Use cpu-clock generates timer interrupts.
Retry at different timer frequencies and skip if no complete pair is captured.
For example, an expected hw int/iret pair is:
hw int 5cc928 => ffff800080010c80 insn: 63 04 00 f1 ... iret ffff800080012284 => 5cc928 insn: e0 03 9f d6
The entry PC is the architectural resume address, here SUBS X3, X3, #1 at 0x5cc928. After the IRQ is handled, ERET returns to the same PC to continue execution.
Assisted-by: Codex:gpt-6 Signed-off-by: Leo Yan leo.yan@arm.com --- tools/perf/tests/builtin-test.c | 1 + tools/perf/tests/shell/coresight/irq_entry_exit.sh | 37 +++++ tools/perf/tests/shell/lib/coresight_exception.sh | 165 +++++++++++++++++++++ tools/perf/tests/tests.h | 1 + tools/perf/tests/workloads/Build | 2 + tools/perf/tests/workloads/branch_not_taken_loop.c | 33 +++++ 6 files changed, 239 insertions(+)
diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c index 6259ed805c5f75799d52ac742b77cf278f98bd9d..8eab221f115e75a3ef90be975d4ed0558ceb2b16 100644 --- a/tools/perf/tests/builtin-test.c +++ b/tools/perf/tests/builtin-test.c @@ -171,6 +171,7 @@ static struct test_workload *workloads[] = { &workload__datasym, &workload__landlock, &workload__traploop, + &workload__branch_not_taken_loop, &workload__inlineloop, &workload__jitdump, &workload__context_switch_loop, diff --git a/tools/perf/tests/shell/coresight/irq_entry_exit.sh b/tools/perf/tests/shell/coresight/irq_entry_exit.sh new file mode 100755 index 0000000000000000000000000000000000000000..f164daa222b0550ceb106aa6a1215740a1432017 --- /dev/null +++ b/tools/perf/tests/shell/coresight/irq_entry_exit.sh @@ -0,0 +1,37 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# CoreSight IRQ entry and exit (exclusive) + +# shellcheck source=../lib/coresight_exception.sh +. "$(dirname "$0")/../lib/coresight_exception.sh" + +test_irq_entry_exit() +{ + # IRQs must return to the interrupted PC. Retry at different timer + # frequencies if trace gaps leave no complete entry/exit pair. + for freq in 1000 4000 10000; do + echo "Recording timer IRQs at $freq Hz" + if ! record_trace_with_cpu_clock "$freq" branch_not_taken_loop; then + cat "$tmpdir/record.log" + echo "Failed to record the IRQ workload" + return 1 + fi + + decode_trace || return 1 + + check_exception_pairs "hw int" "iret" 0 + result=$? + case $result in + 0) return 0 ;; + 1) cat "$tmpdir/script.log"; return 1 ;; + 2) echo "No complete IRQ pair in this recording" ;; + *) return "$result" ;; + esac + done + + echo "[Skip] No complete IRQ pair after three recordings" + return 2 +} + +setup_exception_test || exit $? +test_irq_entry_exit diff --git a/tools/perf/tests/shell/lib/coresight_exception.sh b/tools/perf/tests/shell/lib/coresight_exception.sh new file mode 100644 index 0000000000000000000000000000000000000000..f0133c258625147405ba4641ff7e442437fc66b8 --- /dev/null +++ b/tools/perf/tests/shell/lib/coresight_exception.sh @@ -0,0 +1,165 @@ +# SPDX-License-Identifier: GPL-2.0 +# Common helpers for CoreSight exception entry and return tests. + +cleanup() +{ + case $? in + 0|2) rm -rf "$tmpdir" ;; + *) echo "Test files retained in $tmpdir" ;; + esac +} + +record_trace_with_cpu_clock() +{ + local clock_opts=() + + record_freq=$1 + shift + + # A zero frequency omits the cpu-clock event. + if [ "$record_freq" -gt 0 ]; then + # Generate timer IRQs without delivering a signal to the workload. + clock_opts=(-e cpu-clock:u -F "$record_freq") + fi + + # FIFO control bounds the trace to the workload. + # --kcore supplies the running kernel's instructions for decoding. + taskset -c "$cpu" perf record -B --no-bpf-event --per-thread --kcore \ + -e cs_etm/timestamp=0/uk "${clock_opts[@]}" \ + -m,4M -D -1 --control fifo:"$tmpdir/ctl","$tmpdir/ack" \ + -o "$tmpdir/data" -- \ + perf test --record-ctl fifo:"$tmpdir/ctl","$tmpdir/ack" -w "$@" \ + > "$tmpdir/record.log" 2>&1 +} + +record_trace() +{ + record_trace_with_cpu_clock 0 "$@" +} + +decode_trace() +{ + local sw_fields=() + + # perf script rejects -F sw: when no software event was recorded. + if [ "$record_freq" -gt 0 ]; then + sw_fields=(-F sw:) + fi + + if ! perf script -i "$tmpdir/data" --itrace=b \ + -F hw:ip,addr,flags,insn "${sw_fields[@]}" \ + > "$tmpdir/script" 2> "$tmpdir/script.log"; then + cat "$tmpdir/script.log" + echo "Failed to decode the exception trace" + return 1 + fi +} + +# Arguments: entry pattern, exit pattern, expected return PC - entry PC. +# Return 2 if no complete pair survives the trace gaps. +check_exception_pairs() +{ + local difference=$3 entry_pc resume_pc pairs=0 + + awk -v entry_pattern="$1" -v exit_pattern="$2" ' + function address(pc) { + sub(/^0[xX]/, "", pc) + sub(/^0+/, "", pc) + return "0x" (pc == "" ? "0" : tolower(pc)) + } + BEGIN { + entry_pattern = "^[[:space:]]*(" entry_pattern ")([[:space:]]|$)" + exit_pattern = "^[[:space:]]*(" exit_pattern ")([[:space:]]|$)" + } + NF { + # A trace boundary invalidates all pending entries, including nested ones. + if ($1 == "tr" && ($2 == "strt" || $2 == "end")) { + depth = 0 + next + } + + is_entry = ($1 == "hw" && $2 == "int") || $1 == "int" || $1 == "syscall" + is_exit = $1 == "iret" || $1 == "sysret" + if (!is_entry && !is_exit) + next + + # Branch fields are: flags source => destination [insn: bytes]. + arrow = 0 + for (i = 1; i <= NF; i++) { + if ($i == "=>") { + arrow = i + break + } + } + if (arrow < 3 || arrow == NF) { + print "FAIL: Unexpected perf script output: " $0 > "/dev/stderr" + errors++ + depth = 0 + next + } + + # Track every exception so an unrelated nested IRET cannot close + # a selected outer entry. Match the requested flags at each depth. + if (is_entry) { + entry_pc[++depth] = address($(arrow - 1)) + selected[depth] = $0 ~ entry_pattern + next + } + if (!depth) + next + + if (selected[depth] && $0 ~ exit_pattern) + print entry_pc[depth], address($(arrow + 1)) + depth-- + } + END { + if (errors) + exit 1 + }' "$tmpdir/script" > "$tmpdir/exception_pairs" || return 1 + + # Bash uses integer arithmetic, preserving all bits of 64-bit PCs. + while read -r entry_pc resume_pc; do + if [ "$((resume_pc - entry_pc))" -ne "$difference" ]; then + printf "FAIL: Entry %s returns to %s, expected PC difference %d\n" \ + "$entry_pc" "$resume_pc" "$difference" + return 1 + fi + pairs=$((pairs + 1)) + done < "$tmpdir/exception_pairs" + + if [ "$pairs" -eq 0 ]; then + echo "No complete exception entry/exit pair" + return 2 + fi + printf "Checked %d exception pairs with PC difference %d\n" "$pairs" "$difference" +} + +setup_exception_test() +{ + [ "$(uname -m)" = aarch64 ] || return 2 + perf check feature -q libopencsd || return 2 + [ -d /sys/bus/event_source/devices/cs_etm ] || return 2 + + export LC_ALL=C + command -v taskset >/dev/null 2>&1 || return 2 + + tmpdir=$(mktemp -d /tmp/perf-cs-etm-exception.XXXXXX) || return 1 + trap cleanup EXIT + trap 'exit 1' HUP INT TERM + mkfifo "$tmpdir/ctl" "$tmpdir/ack" || return 1 + + # Keep each exception entry and return in the same trace queue. + cpu=$(awk '/Cpus_allowed_list:/ { + split($2, cpus, /[-,]/) + print cpus[1] + }' /proc/self/status) + + # Check permissions, FIFO control and the trace sink before testing. + if ! record_trace callchain; then + cat "$tmpdir/record.log" + echo "[Skip] Cannot record user/kernel CoreSight trace with --kcore" + return 2 + fi + + return 0 +} diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h index b2520a564417b61718f1dcb23da1f8cc0f601906..0a61129b31ab6997616f7d04d29af73961f4f861 100644 --- a/tools/perf/tests/tests.h +++ b/tools/perf/tests/tests.h @@ -246,6 +246,7 @@ DECLARE_WORKLOAD(brstack); DECLARE_WORKLOAD(datasym); DECLARE_WORKLOAD(landlock); DECLARE_WORKLOAD(traploop); +DECLARE_WORKLOAD(branch_not_taken_loop); DECLARE_WORKLOAD(inlineloop); DECLARE_WORKLOAD(jitdump); DECLARE_WORKLOAD(context_switch_loop); diff --git a/tools/perf/tests/workloads/Build b/tools/perf/tests/workloads/Build index 048e371eb63e316453b6b46ebd0a02794c3d25d7..ca22a82073ed08de5d4a27d0a3b33ddcba2e3f3e 100644 --- a/tools/perf/tests/workloads/Build +++ b/tools/perf/tests/workloads/Build @@ -9,6 +9,7 @@ perf-test-y += brstack.o perf-test-y += datasym.o perf-test-y += landlock.o perf-test-y += traploop.o +perf-test-y += branch_not_taken_loop.o perf-test-y += inlineloop.o perf-test-y += jitdump.o perf-test-y += context_switch_loop.o @@ -25,6 +26,7 @@ CFLAGS_leafloop.o = -g -O0 -fno-inline -fno-omit-frame-pointer -U_FORTIF CFLAGS_brstack.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE CFLAGS_datasym.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE CFLAGS_traploop.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE +CFLAGS_branch_not_taken_loop.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE CFLAGS_inlineloop.o = -g -O2 CFLAGS_deterministic.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE CFLAGS_named_threads.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE diff --git a/tools/perf/tests/workloads/branch_not_taken_loop.c b/tools/perf/tests/workloads/branch_not_taken_loop.c new file mode 100644 index 0000000000000000000000000000000000000000..6376b1c476e2f4814a94c91a5321ae919a818826 --- /dev/null +++ b/tools/perf/tests/workloads/branch_not_taken_loop.c @@ -0,0 +1,33 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/compiler.h> +#include "../tests.h" + +#ifdef __aarch64__ +static void branch_not_taken(void) +{ + /* Keep x1 > x2 so B.LS is never taken. */ + asm volatile( + "mov x1, #2\n" + "mov x2, #1\n" + "movz x3, #0xffff\n" + "movk x3, #0x0080, lsl #16\n" + "1: cmp x1, x2\n" + ".Ltest_branch:\n" + "b.ls 2f\n" + ".Lfallthrough:\n" + "subs x3, x3, #1\n" + "b.ne 1b\n" + "2:\n" + : : : "x1", "x2", "x3", "cc"); +} +#else +static void branch_not_taken(void) { } +#endif + +static int branch_not_taken_loop(int argc __maybe_unused, const char **argv __maybe_unused) +{ + branch_not_taken(); + return 0; +} + +DEFINE_WORKLOAD(branch_not_taken_loop);
On 23/09/2026 16:21, Leo Yan wrote:
Add an AArch64 branch_not_taken_loop workload and a CoreSight shell test checking IRQ entry and exit PCs. Use cpu-clock generates timer interrupts.
Retry at different timer frequencies and skip if no complete pair is captured.
Maybe we need to look into storing existing recordings somewhere or committing small ones into the repo again. This test took 3 minutes to run on my N1SDP, which is 3x longer than the entire Coresight suite, and then it skipped anyway:
173: CoreSight IRQ entry and exit: ---- start ---- test child forked, pid 6804 Recording timer IRQs at 1000 Hz No complete exception entry/exit pair No complete IRQ pair in this recording Recording timer IRQs at 4000 Hz No complete exception entry/exit pair No complete IRQ pair in this recording Recording timer IRQs at 10000 Hz No complete exception entry/exit pair No complete IRQ pair in this recording [Skip] No complete IRQ pair after three recordings ---- end(-2) ---- 173: CoreSight IRQ entry and exit
Really this test is only looking for how a single exception on a single branch is decoded, it shouldn't be affected by how it was recorded. Even a few bytes of trace would be enough to test it. Or maybe we could add some kind of unit test framework that mocks out OpenCSD and allows us to script how different packets affect the decoder?
But we have to be careful to not add slow tests because it means they're much less likely to be run, or results ignored as it's so hard to iterate quickly. Or also in this case, silently skipped if something regresses.
For example, an expected hw int/iret pair is:
hw int 5cc928 => ffff800080010c80 insn: 63 04 00 f1 ... iret ffff800080012284 => 5cc928 insn: e0 03 9f d6
The entry PC is the architectural resume address, here SUBS X3, X3, #1 at 0x5cc928. After the IRQ is handled, ERET returns to the same PC to continue execution.
Assisted-by: Codex:gpt-6 Signed-off-by: Leo Yan leo.yan@arm.com
tools/perf/tests/builtin-test.c | 1 + tools/perf/tests/shell/coresight/irq_entry_exit.sh | 37 +++++ tools/perf/tests/shell/lib/coresight_exception.sh | 165 +++++++++++++++++++++ tools/perf/tests/tests.h | 1 + tools/perf/tests/workloads/Build | 2 + tools/perf/tests/workloads/branch_not_taken_loop.c | 33 +++++ 6 files changed, 239 insertions(+)
diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c index 6259ed805c5f75799d52ac742b77cf278f98bd9d..8eab221f115e75a3ef90be975d4ed0558ceb2b16 100644 --- a/tools/perf/tests/builtin-test.c +++ b/tools/perf/tests/builtin-test.c @@ -171,6 +171,7 @@ static struct test_workload *workloads[] = { &workload__datasym, &workload__landlock, &workload__traploop,
- &workload__branch_not_taken_loop, &workload__inlineloop, &workload__jitdump, &workload__context_switch_loop,
diff --git a/tools/perf/tests/shell/coresight/irq_entry_exit.sh b/tools/perf/tests/shell/coresight/irq_entry_exit.sh new file mode 100755 index 0000000000000000000000000000000000000000..f164daa222b0550ceb106aa6a1215740a1432017 --- /dev/null +++ b/tools/perf/tests/shell/coresight/irq_entry_exit.sh @@ -0,0 +1,37 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# CoreSight IRQ entry and exit (exclusive)
+# shellcheck source=../lib/coresight_exception.sh
No need for these anymore, see https://lore.kernel.org/linux-perf-users/20260921-james-perf-shellcheck-sour...
+. "$(dirname "$0")/../lib/coresight_exception.sh"
+test_irq_entry_exit() +{
- # IRQs must return to the interrupted PC. Retry at different timer
- # frequencies if trace gaps leave no complete entry/exit pair.
- for freq in 1000 4000 10000; do
echo "Recording timer IRQs at $freq Hz"if ! record_trace_with_cpu_clock "$freq" branch_not_taken_loop; thencat "$tmpdir/record.log"echo "Failed to record the IRQ workload"return 1fidecode_trace || return 1check_exception_pairs "hw int" "iret" 0result=$?case $result in0) return 0 ;;1) cat "$tmpdir/script.log"; return 1 ;;2) echo "No complete IRQ pair in this recording" ;;*) return "$result" ;;esac- done
- echo "[Skip] No complete IRQ pair after three recordings"
- return 2
+}
+setup_exception_test || exit $? +test_irq_entry_exit diff --git a/tools/perf/tests/shell/lib/coresight_exception.sh b/tools/perf/tests/shell/lib/coresight_exception.sh new file mode 100644 index 0000000000000000000000000000000000000000..f0133c258625147405ba4641ff7e442437fc66b8 --- /dev/null +++ b/tools/perf/tests/shell/lib/coresight_exception.sh @@ -0,0 +1,165 @@ +# SPDX-License-Identifier: GPL-2.0 +# Common helpers for CoreSight exception entry and return tests.
+cleanup() +{
- case $? in
- 0|2) rm -rf "$tmpdir" ;;
- *) echo "Test files retained in $tmpdir" ;;
- esac
+}
+record_trace_with_cpu_clock() +{
- local clock_opts=()
- record_freq=$1
- shift
- # A zero frequency omits the cpu-clock event.
- if [ "$record_freq" -gt 0 ]; then
# Generate timer IRQs without delivering a signal to the workload.clock_opts=(-e cpu-clock:u -F "$record_freq")- fi
- # FIFO control bounds the trace to the workload.
- # --kcore supplies the running kernel's instructions for decoding.
- taskset -c "$cpu" perf record -B --no-bpf-event --per-thread --kcore \
-e cs_etm/timestamp=0/uk "${clock_opts[@]}" \-m,4M -D -1 --control fifo:"$tmpdir/ctl","$tmpdir/ack" \-o "$tmpdir/data" -- \perf test --record-ctl fifo:"$tmpdir/ctl","$tmpdir/ack" -w "$@" \> "$tmpdir/record.log" 2>&1
There's a shellcheck warning here:
(warning): Word is of the form "A"B"C" (B indicated). Did you mean "ABC" or "A"B"C"?
You can avoid it by passing the whole path as "$cf" and "$af" like the other coresight tests.
Record the existing callchain workload, which issues gettid() through SVC. The test requires syscall/sysret pairs to advance the PC by four bytes.
For example, an expected syscall/sysret pair is:
syscall bb76e4 => ffff800080010c00 insn: 01 00 00 d4 ... sysret ffff800080012284 => bb76e8 insn: e0 03 9f d6
The entry instruction is SVC #0 at 0xbb76e4. On syscall completion, ERET returns to 0xbb76e8, the instruction immediately after SVC.
Assisted-by: Codex:gpt-6 Signed-off-by: Leo Yan leo.yan@arm.com --- .../tests/shell/coresight/syscall_entry_exit.sh | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+)
diff --git a/tools/perf/tests/shell/coresight/syscall_entry_exit.sh b/tools/perf/tests/shell/coresight/syscall_entry_exit.sh new file mode 100755 index 0000000000000000000000000000000000000000..bf61b1fdf6f15e290295afc7577870c6864ddd93 --- /dev/null +++ b/tools/perf/tests/shell/coresight/syscall_entry_exit.sh @@ -0,0 +1,21 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# CoreSight syscall entry and exit (exclusive) + +# shellcheck source=../lib/coresight_exception.sh +. "$(dirname "$0")/../lib/coresight_exception.sh" + +test_syscall_entry_exit() +{ + # callchain() invokes gettid() through SVC, which returns to PC + 4. + if ! record_trace callchain; then + cat "$tmpdir/record.log" + echo "Failed to record the callchain workload" + return 1 + fi + decode_trace || return 1 + check_exception_pairs "syscall" "sysret" 4 || return 1 +} + +setup_exception_test || exit $? +test_syscall_entry_exit
Add page_fault_loop to repeatedly discard an anonymous page with MADV_DONTNEED and read it again. Each demand fault is resolved without signal delivery, so the load must be retried at the same PC.
For example, an expected int/iret pair is:
int 5ccadc => ffff800080010c00 insn: 00 00 40 39 ... iret ffff800080012284 => 5ccadc insn: e0 03 9f d6
The LDRB W0, [X0] at 0x5ccadc faults before completing. After resolving the page fault, the kernel returns via ERET to the same PC to retry it.
Assisted-by: Codex:gpt-6 Signed-off-by: Leo Yan leo.yan@arm.com --- tools/perf/tests/builtin-test.c | 1 + .../perf/tests/shell/coresight/abort_entry_exit.sh | 21 ++++++++++++ tools/perf/tests/tests.h | 1 + tools/perf/tests/workloads/Build | 2 ++ tools/perf/tests/workloads/page_fault_loop.c | 37 ++++++++++++++++++++++ 5 files changed, 62 insertions(+)
diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c index 8eab221f115e75a3ef90be975d4ed0558ceb2b16..1af8dfd45d1a2ff5c3206234cd75fb173e122d88 100644 --- a/tools/perf/tests/builtin-test.c +++ b/tools/perf/tests/builtin-test.c @@ -172,6 +172,7 @@ static struct test_workload *workloads[] = { &workload__landlock, &workload__traploop, &workload__branch_not_taken_loop, + &workload__page_fault_loop, &workload__inlineloop, &workload__jitdump, &workload__context_switch_loop, diff --git a/tools/perf/tests/shell/coresight/abort_entry_exit.sh b/tools/perf/tests/shell/coresight/abort_entry_exit.sh new file mode 100755 index 0000000000000000000000000000000000000000..483cf39b3966e89591d4e06d03a3fc6709fc3dc9 --- /dev/null +++ b/tools/perf/tests/shell/coresight/abort_entry_exit.sh @@ -0,0 +1,21 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# CoreSight abort entry and exit (exclusive) + +# shellcheck source=../lib/coresight_exception.sh +. "$(dirname "$0")/../lib/coresight_exception.sh" + +test_abort_entry_exit() +{ + # A demand page fault retries the load at the same PC after resolution. + if ! record_trace page_fault_loop; then + cat "$tmpdir/record.log" + echo "Failed to record the page fault workload" + return 1 + fi + decode_trace || return 1 + check_exception_pairs "int" "iret" 0 || return 1 +} + +setup_exception_test || exit $? +test_abort_entry_exit diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h index 0a61129b31ab6997616f7d04d29af73961f4f861..0fc02c5b060133ef27b80de5c868d798d65a7c53 100644 --- a/tools/perf/tests/tests.h +++ b/tools/perf/tests/tests.h @@ -247,6 +247,7 @@ DECLARE_WORKLOAD(datasym); DECLARE_WORKLOAD(landlock); DECLARE_WORKLOAD(traploop); DECLARE_WORKLOAD(branch_not_taken_loop); +DECLARE_WORKLOAD(page_fault_loop); DECLARE_WORKLOAD(inlineloop); DECLARE_WORKLOAD(jitdump); DECLARE_WORKLOAD(context_switch_loop); diff --git a/tools/perf/tests/workloads/Build b/tools/perf/tests/workloads/Build index ca22a82073ed08de5d4a27d0a3b33ddcba2e3f3e..e18fdaa6fd1927ed406e703a7ecbcc19a111caf3 100644 --- a/tools/perf/tests/workloads/Build +++ b/tools/perf/tests/workloads/Build @@ -10,6 +10,7 @@ perf-test-y += datasym.o perf-test-y += landlock.o perf-test-y += traploop.o perf-test-y += branch_not_taken_loop.o +perf-test-y += page_fault_loop.o perf-test-y += inlineloop.o perf-test-y += jitdump.o perf-test-y += context_switch_loop.o @@ -27,6 +28,7 @@ CFLAGS_brstack.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE CFLAGS_datasym.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE CFLAGS_traploop.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE CFLAGS_branch_not_taken_loop.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE +CFLAGS_page_fault_loop.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE CFLAGS_inlineloop.o = -g -O2 CFLAGS_deterministic.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE CFLAGS_named_threads.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE diff --git a/tools/perf/tests/workloads/page_fault_loop.c b/tools/perf/tests/workloads/page_fault_loop.c new file mode 100644 index 0000000000000000000000000000000000000000..4afbc4ab0a55400360facdc3a99ea5a5a9d52596 --- /dev/null +++ b/tools/perf/tests/workloads/page_fault_loop.c @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/compiler.h> +#include <stdio.h> +#include <sys/mman.h> +#include <unistd.h> +#include "../tests.h" + +static int page_fault_loop(int argc __maybe_unused, const char **argv __maybe_unused) +{ + size_t size = getpagesize(); + char *page; + int ret = 1; + + page = mmap(NULL, size, PROT_READ, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (page == MAP_FAILED) { + perror("mmap"); + return 1; + } + + for (int i = 0; i < 256; i++) { + /* Drop the PTE so the kernel must resolve a fault on each read. */ + if (madvise(page, size, MADV_DONTNEED)) { + perror("madvise"); + goto out; + } + if (READ_ONCE(*page)) { + fprintf(stderr, "Anonymous page is not zero-filled\n"); + goto out; + } + } + ret = 0; +out: + munmap(page, size); + return ret; +} + +DEFINE_WORKLOAD(page_fault_loop);
Use traploop's EL0 read of ID_AA64ISAR0_EL1 to check an emulated instruction. The kernel emulates MRS and advances the return PC by four bytes.
Select MRS entries by instruction encoding because page faults also appear as int/iret pairs but return to the same PC. Reuse the common FIFO recorder and pair checker.
For example, an expected int/iret pair is:
int 5cc808 => ffff800080010c00 insn: 00 06 38 d5 ... iret ffff800080012284 => 5cc80c insn: e0 03 9f d6
The entry instruction is MRS X0, ID_AA64ISAR0_EL1. After emulating the read at 0x5cc808, the kernel returns via ERET to the next instruction at 0x5cc80c.
Assisted-by: Codex:gpt-6 Signed-off-by: Leo Yan leo.yan@arm.com --- .../perf/tests/shell/coresight/trap_entry_exit.sh | 24 ++++++++++++++++++++++ 1 file changed, 24 insertions(+)
diff --git a/tools/perf/tests/shell/coresight/trap_entry_exit.sh b/tools/perf/tests/shell/coresight/trap_entry_exit.sh new file mode 100755 index 0000000000000000000000000000000000000000..3ebd4033b247a1a43e0139868972afd41ba905ba --- /dev/null +++ b/tools/perf/tests/shell/coresight/trap_entry_exit.sh @@ -0,0 +1,24 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# CoreSight emulated instruction entry and exit (exclusive) + +# shellcheck source=../lib/coresight_exception.sh +. "$(dirname "$0")/../lib/coresight_exception.sh" + +test_trap_entry_exit() +{ + # traploop reads ID_AA64ISAR0_EL1 from EL0. The kernel emulates MRS and + # advances the saved PC, so IRET must resume four bytes after the trap. + if ! record_trace traploop 256; then + cat "$tmpdir/record.log" + echo "Failed to record the traploop workload" + return 1 + fi + decode_trace || return 1 + # Page faults also appear as "int", but retry the same PC. Select MRS + # Xt, ID_AA64ISAR0_EL1 by its encoding; the low five bits select Xt. + check_exception_pairs "int.*insn: [01][[:xdigit:]] 06 38 d5" "iret" 4 || return 1 +} + +setup_exception_test || exit $? +test_trap_entry_exit
On 23/09/2026 16:21, Leo Yan wrote:
CoreSight currently models exception entry by changing the preceding instruction range into a taken branch. This can give an IRQ the source PC of an instruction that already retired and overwrite a real branch immediately before the exception.
For an untaken B.LS followed by an IRQ, perf script currently reports:
hw int 4000f8 => ffff800080010c80 b.ls #0x400118 ... iret ffff800080012284 => 4000fc eret
The hardware trace supplies 0x4000fc as the preferred return address. For an IRQ, this is the architectural resume PC (Arm ARM, R_VBQMV). B.LS has retired, and the saved PC identifies the boundary before the following MOV. Using this PC as the IRQ source reflects the architectural state at exception entry:
hw int 4000fc => ffff800080010c80 movz x2, #0x1796 ... iret ffff800080012284 => 4000fc eret
The IRQ sample represents the transfer from this architectural execution position to the handler. The series synthesizes exception entries from exception packets, preserving the preceding branch and its outcome.
The supporting changes:
- Let decoders supply sample.ret_addr so later instruction fetching cannot change thread-stack return addresses. Apply this to Intel PT asynchronous samples as well.
- Prepare packet ISA and instruction-size handling and share sample synthesis helpers. Mark untaken branches and break history when instruction memory is unavailable.
- Add a thread-stack regression test and four AArch64 CoreSight tests. IRQs and page faults must resume at the entry PC; SVC and emulated MRS must resume four bytes later.
Based on the AI search and test on my x86 machine, this matches perf's Intel PT handling of IRQs. Intel PT records the next instruction's IP in the FUP packet, and Perf uses that IP as the interrupt sample's source.
This series is verified on Orion6 board with "perf test coresight".
Signed-off-by: Leo Yan leo.yan@arm.com
Changes in v2:
- Rework the fix around exception packets to preserve both exception entries and preceding branches.
- Add explicit return addresses and the Intel PT asynchronous-branch fix.
- Split out packet/synthesis preparation, record not-taken branches and handle unreadable instruction memory.
- Add thread-stack regression coverage and four CoreSight tests.
- Link to v1: https://lore.kernel.org/r/20260713-perf_cs_etm_fix_non_taken-v1-0-4561607fc6...
Leo Yan (14): perf sample: Allow decoders to supply branch return addresses perf intel-pt: Preserve return addresses for asynchronous branches perf cs-etm: Break branch history when instruction memory is unavailable perf cs-etm: Centralize packet ISA initialization perf cs-etm: Use the recorded instruction size for A32 and A64 perf cs-etm: Mark branches that were not taken perf cs-etm: Factor out final instruction sample synthesis perf cs-etm: Centralize branch sample synthesis checks perf cs-etm: Classify exception calls using the exception packet perf cs-etm: Synthesize exception entries separately from branches perf tests: Check CoreSight IRQ entry and exit perf tests: Check CoreSight syscall entry and exit perf tests: Check CoreSight abort entry and exit perf tests: Check CoreSight emulated instruction entry and exit
tools/perf/tests/Build | 1 + tools/perf/tests/builtin-test.c | 3 + .../perf/tests/shell/coresight/abort_entry_exit.sh | 21 ++ tools/perf/tests/shell/coresight/irq_entry_exit.sh | 37 ++++ .../tests/shell/coresight/syscall_entry_exit.sh | 21 ++ .../perf/tests/shell/coresight/trap_entry_exit.sh | 24 +++ tools/perf/tests/shell/lib/coresight_exception.sh | 165 +++++++++++++++ tools/perf/tests/tests.h | 3 + tools/perf/tests/thread-stack.c | 106 ++++++++++ tools/perf/tests/workloads/Build | 4 + tools/perf/tests/workloads/branch_not_taken_loop.c | 33 +++ tools/perf/tests/workloads/page_fault_loop.c | 37 ++++ tools/perf/util/cs-etm-decoder/cs-etm-decoder.c | 106 +++++++--- tools/perf/util/cs-etm.c | 222 ++++++++++----------- tools/perf/util/cs-etm.h | 2 + tools/perf/util/intel-pt.c | 7 + tools/perf/util/sample.c | 1 + tools/perf/util/sample.h | 5 + tools/perf/util/thread-stack.c | 5 +- 19 files changed, 656 insertions(+), 147 deletions(-)
base-commit: edd8a9fe2eca009599e013a29c421c7a6b5ad1b9 change-id: 20260713-perf_cs_etm_fix_non_taken-5b4d7f73f41e
Best regards,
Reviewed-by: James Clark james.clark@linaro.org
-
James Clark -
Leo Yan