Introduction of TPDM CMB(Continuous Multi Bit) subunit
CMB subunit is responsible for creating a dataset element, and is also
optionally responsible for packing it to fit multiple elements on a
single ATB transfer if possible in the configuration. The TPDM Core
Datapath requests timestamps be stored by the TPDA and then delivering
ATB sized data (depending on ATB width and element size, this could
be smaller or larger than a dataset element) to the ATB Mast FSM.
The CMB makes trace elements in two modes. In �continuous� mode, every
valid data cycle creates an element. In �trace on change� mode, when
valid data changes on the bus, a trace element is created. In
continuous mode, all cycles where this condition is true create trace
elements. In trace on change mode, a data element is only when the
previously sampled input is different from the current sampled input.
The CMB subunit must be configured prior to enablement. This series
adds support for TPDM to configure the configure CMB subunit.
Once this series patches are applied properly, the new tpdm nodes for
should be observed at the tpdm path /sys/bus/coresight/devices/tpdm*
which supports CMB subunit.
e.g.
root@qemuarm64:/sys/devices/platform/soc@0/684c000.tpdm/tpdm0# ls -l
-rw-r--r-- 1 root root 4096 Jan 1 00:00 cmb_mode
drwxr-xr-x 2 root root 0 Jan 1 00:00 cmb_msr
drwxr-xr-x 2 root root 0 Jan 1 00:00 cmb_patt
drwxr-xr-x 2 root root 0 Jan 1 00:00 cmb_trig_patt
-rw-r--r-- 1 root root 4096 Jan 1 00:00 cmb_trig_ts
-rw-r--r-- 1 root root 4096 Jan 1 00:00 cmb_ts_all
drwxr-xr-x 2 root root 0 Jan 1 00:00 connections
drwxr-xr-x 2 root root 0 Jan 1 00:00 dsb_edge
drwxr-xr-x 2 root root 0 Jan 1 00:00 dsb_msr
drwxr-xr-x 2 root root 0 Jan 1 00:00 dsb_patt
drwxr-xr-x 2 root root 0 Jan 1 00:00 dsb_trig_patt
-rw-r--r-- 1 root root 4096 Jan 1 00:00 enable_source
--w------- 1 root root 4096 Jan 1 00:00 integration_test
drwxr-xr-x 2 root root 0 Ja? 1 00:00 power
--w------- 1 root root 4096 Jan 1 00:00 reset_dataset
lrwxrwxrwx 1 root root 0 Apr 5 2021 subsystem -> ../../../../../bus/coresight
-rw-r--r-- 1 root root 4096 Apr 5 2021 uevent
-r--r--r-- 1 root root 4096 Jan 1 00:00 waiting_for_supplier
We can use the commands are similar to the below to configure the
TPDMs which support CMB subunit. Enable coresight sink first.
echo 1 > /sys/bus/coresight/devices/tmc_etf0/enable_sink
echo 1 > /sys/bus/coresight/devices/tpdm0/reset_dataset
echo 1 > /sys/bus/coresight/devices/tpdm0/cmb_mode
echo 1 > /sys/bus/coresight/devices/tpdm0/cmb_patt/enable_ts
echo 0xFFFFFFFF > /sys/bus/coresight/devices/tpdm0/cmb_patt/tpmr0
echo 0 > /sys/bus/coresight/devices/tpdm0/cmb_trig_ts
echo 0xFFFFFFFF > /sys/bus/coresight/devices/tpdm0/cmb_trig_patt/xpr1
echo 1 > /sys/bus/coresight/devices/tpdm0/enable_source
codelinaro link:
https://git.codelinaro.org/clo/linux-kernel/coresight/-/commits/tpdm-cmb-v3
Changes in V3:
1. Add 8-bit support to the description in the TPDM devicetree document.
-- Rob Herring
2. Change how the result is produced in "tpdm_read_element_size".
-- James Clark
3. Calling "tpdm_clear_element_size" at the beginning of
"tpda_enable_port".
-- James Clark
4. Use "dsb_esize" and "cmb_esize" to determine whether multiple TPDMs
are detected on a TPDA input port in "tpda_get_element_size".
-- James Clark
5. Modify the judgment logic in "tpda_enable_port".
-- James Clark
6. Add more description of "cmb_mode" to TPDM SysFS document.
-- James Clark
Changes in V2:
1. Optimizate and modify this patch series based on the patch series
"Add support to configure TPDM CMB subunit".
2. Modify the functions that read the element size of DSB/CMB in TPDA driver.
Tao Zhang (8):
dt-bindings: arm: Add support for CMB element size
coresight-tpda: Add support to configure CMB element
coresight-tpdm: Add CMB dataset support
coresight-tpdm: Add support to configure CMB
coresight-tpdm: Add pattern registers support for CMB
coresight-tpdm: Add timestamp control register support for the CMB
dt-bindings: arm: Add support for TPDM CMB MSR register
coresight-tpdm: Add msr register support for CMB
.../testing/sysfs-bus-coresight-devices-tpdm | 87 ++++
.../bindings/arm/qcom,coresight-tpdm.yaml | 38 ++
drivers/hwtracing/coresight/coresight-tpda.c | 117 +++---
drivers/hwtracing/coresight/coresight-tpda.h | 6 +
drivers/hwtracing/coresight/coresight-tpdm.c | 390 +++++++++++++++++-
drivers/hwtracing/coresight/coresight-tpdm.h | 87 ++++
6 files changed, 673 insertions(+), 52 deletions(-)
--
2.17.1
Hi Greg
Please find fixes for hwtracing/coresight subsystem, targetting Linux v6.7.
Kindly pull
Suzuki
The following changes since commit b85ea95d086471afb4ad062012a4d73cd328fa86:
Linux 6.7-rc1 (2023-11-12 16:19:07 -0800)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/coresight/linux.git tags/coresight-fixes-for-v6.7-rc1
for you to fetch changes up to 862c135bde8bc185e8aae2110374175e6a1b6ed5:
coresight: ultrasoc-smb: Fix uninitialized before use buf_hw_base (2023-11-16 10:00:14 +0000)
----------------------------------------------------------------
coresight: Fixes for v6.7-rc1
Here are a few fixes for the hwtracing subsystem targetting v6.7.
Includes:
- Ultrasoc-SMB driver fixes
- HiSilicon PTT driver fixes
- Corsight driver fixes
Signed-off-by: Suzuki K Poulose <suzuki.poulose(a)arm.com>
----------------------------------------------------------------
James Clark (1):
coresight: Fix crash when Perf and sysfs modes are used concurrently
Junhao He (4):
hwtracing: hisi_ptt: Add dummy callback pmu::read()
coresight: ultrasoc-smb: Fix sleep while close preempt in enable_smb
coresight: ultrasoc-smb: Config SMB buffer before register sink
coresight: ultrasoc-smb: Fix uninitialized before use buf_hw_base
Uwe Kleine-König (1):
coresight: etm4x: Remove bogous __exit annotation for some functions
Vegard Nossum (1):
Documentation: coresight: fix `make refcheckdocs` warning
Yicong Yang (2):
hwtracing: hisi_ptt: Handle the interrupt in hardirq context
hwtracing: hisi_ptt: Don't try to attach a task
Documentation/trace/coresight/coresight.rst | 2 +-
drivers/hwtracing/coresight/coresight-etm-perf.c | 4 +-
drivers/hwtracing/coresight/coresight-etm4x-core.c | 6 +-
drivers/hwtracing/coresight/ultrasoc-smb.c | 64 +++++++++-------------
drivers/hwtracing/coresight/ultrasoc-smb.h | 6 +-
drivers/hwtracing/ptt/hisi_ptt.c | 14 ++++-
6 files changed, 46 insertions(+), 50 deletions(-)
On 29/11/2023 12:23, Peter Zijlstra wrote:
> On Wed, Nov 29, 2023 at 01:15:43PM +0200, Adrian Hunter wrote:
>> On 29/11/23 12:58, Peter Zijlstra wrote:
>>> On Wed, Nov 29, 2023 at 09:53:39AM +0000, James Clark wrote:
>>>> On 23/11/2023 12:18, Adrian Hunter wrote:
>>>
>>>>> +static void pt_event_pause_resume(struct perf_event *event)
>>>>> +{
>>>>> + if (event->aux_paused)
>>>>> + pt_config_stop(event);
>>>>> + else if (!event->hw.state)
>>>>> + pt_config_start(event);
>>>>> +}
>>>>
>>>> It seems like having a single pause/resume callback rather than separate
>>>> pause and resume ones pushes some of the event state management into the
>>>> individual drivers and would be prone to code duplication and divergent
>>>> behavior.
>>>>
>>>> Would it be possible to move the conditions from here into the core code
>>>> and call separate functions instead?
>>>>
>>>>> +
>>>>> static void pt_event_start(struct perf_event *event, int mode)
>>>>> {
>>>>> struct hw_perf_event *hwc = &event->hw;
>>>>> @@ -1798,6 +1809,7 @@ static __init int pt_init(void)
>>>>> pt_pmu.pmu.del = pt_event_del;
>>>>> pt_pmu.pmu.start = pt_event_start;
>>>>> pt_pmu.pmu.stop = pt_event_stop;
>>>>> + pt_pmu.pmu.pause_resume = pt_event_pause_resume;
>>>>
>>>> The general idea seems ok to me. Is there a reason to not use the
>>>> existing start() stop() callbacks, rather than adding a new one?
>>>>
>>>> I assume it's intended to be something like an optimisation where you
>>>> can turn it on and off without having to do the full setup, teardown and
>>>> emit an AUX record because you know the process being traced never gets
>>>> switched out?
>>>
>>> So the actual scheduling uses ->add() / ->del(), the ->start() /
>>> ->stop() methods are something that can be used after ->add() and before
>>> ->del() to 'temporarily' pause things.
>>>
>>> Pretty much exactly what is required here I think. We currently use this
>>> for PMI throttling and adaptive frequency stuff, but there is no reason
>>> it could not also be used for this.
>>>
>>> As is, we don't track the paused state across ->del() / ->add(), but
>>> perhaps that can be fixed. We can easily add more PERF_EF_ / PERF_HES_
>>> bits to manage things.
>>>
>>>
>>
>> I am not sure stop / start play nice with NMI's from other events e.g.
>>
>> PMC NMI wants to pause or resume AUX but what if AUX event is currently
>> being processed in ->stop() or ->start()? Or maybe that can't happen?
>
> I think that can happen, and pt_event_stop() can actually handle some of
> that, while your pause_resume() thing, which uses pt_config_stop() does
> not.
>
> But yes, I think that if you add pt_event_{stop,start}() calls from
> *other* events their PMI, then you get to deal with more 'fun'.
>
> Something like:
>
> perf_addr_filters_adjust()
> __perf_addr_filters_adjust()
> perf_event_stop()
> __perf_event_stop()
> event->pmu->stop()
> <NMI>
> ...
> perf_event_overflow()
> pt_event->pmu->stop()
> </NMI>
> event->pmu->start() // whoopsie!
>
> Should now be possible.
>
> I think what you want to do is rename pt->handle_nmi into pt->stop_count
> and make it a counter, then ->stop() increments it, and ->start()
> decrements it and everybody ensures the thing doesn't get restart while
> !0 etc..
>
> I suspect you need to guard the generic part of this feature with a new
> PERF_PMU_CAP_ flag and then have the coresight/etc. people opt-in once
> they've audited things.
>
> James, does that work for you?
>
Yep I think that would work.
If I understand it with the stop_count counter, to be able to support
the new CAP, every device would basically have to implement a similar
counter?
Would it be possible to do that reference counting on the outside of
start() and stop() in the core code? So that a device only ever sees one
call to start and one call to stop and doesn't need to do any extra
tracking?
On 23/11/2023 12:18, Adrian Hunter wrote:
> Prevent tracing to start if aux_paused.
>
> Implement pause_resume() callback. When aux_paused, stop tracing. When
> not aux_paused, only start tracing if it isn't currently meant to be
> stopped.
>
> Signed-off-by: Adrian Hunter <adrian.hunter(a)intel.com>
> ---
> arch/x86/events/intel/pt.c | 12 ++++++++++++
> 1 file changed, 12 insertions(+)
>
> diff --git a/arch/x86/events/intel/pt.c b/arch/x86/events/intel/pt.c
> index 42a55794004a..aa883b64814a 100644
> --- a/arch/x86/events/intel/pt.c
> +++ b/arch/x86/events/intel/pt.c
> @@ -418,6 +418,9 @@ static void pt_config_start(struct perf_event *event)
> struct pt *pt = this_cpu_ptr(&pt_ctx);
> u64 ctl = event->hw.config;
>
> + if (event->aux_paused)
> + return;
> +
> ctl |= RTIT_CTL_TRACEEN;
> if (READ_ONCE(pt->vmx_on))
> perf_aux_output_flag(&pt->handle, PERF_AUX_FLAG_PARTIAL);
> @@ -1563,6 +1566,14 @@ EXPORT_SYMBOL_GPL(intel_pt_handle_vmx);
> * PMU callbacks
> */
>
> +static void pt_event_pause_resume(struct perf_event *event)
> +{
> + if (event->aux_paused)
> + pt_config_stop(event);
> + else if (!event->hw.state)
> + pt_config_start(event);
> +}
It seems like having a single pause/resume callback rather than separate
pause and resume ones pushes some of the event state management into the
individual drivers and would be prone to code duplication and divergent
behavior.
Would it be possible to move the conditions from here into the core code
and call separate functions instead?
> +
> static void pt_event_start(struct perf_event *event, int mode)
> {
> struct hw_perf_event *hwc = &event->hw;
> @@ -1798,6 +1809,7 @@ static __init int pt_init(void)
> pt_pmu.pmu.del = pt_event_del;
> pt_pmu.pmu.start = pt_event_start;
> pt_pmu.pmu.stop = pt_event_stop;
> + pt_pmu.pmu.pause_resume = pt_event_pause_resume;
The general idea seems ok to me. Is there a reason to not use the
existing start() stop() callbacks, rather than adding a new one?
I assume it's intended to be something like an optimisation where you
can turn it on and off without having to do the full setup, teardown and
emit an AUX record because you know the process being traced never gets
switched out?
Could you make it so that it works out of the box, with the option of
later optimisation if you do something like this (not here but something
like this in events/core.c):
/* Use specialised pause/resume if it exists, otherwise use more
* expensive start/stop.
*/
if (pmu->pause_resume)
pmu->pause_resume(...)
else
pmu->stop(...)
> pt_pmu.pmu.snapshot_aux = pt_event_snapshot_aux;
> pt_pmu.pmu.read = pt_event_read;
> pt_pmu.pmu.setup_aux = pt_buffer_setup_aux;
This is a combination of the RFC for nVHE here [1] and v3 of VHE version
here [2]. After a few of the review comments it seemed much simpler for
both versions to use the same interface and be in the same patchset.
FEAT_TRF is a Coresight feature that allows trace capture to be
completely filtered at different exception levels, unlike the existing
TRCVICTLR controls which may still emit target addresses of branches,
even if the following trace is filtered.
Without FEAT_TRF, it was possible to start a trace session on a host and
also collect trace from the guest as TRCVICTLR was never programmed to
exclude guests (and it could still emit target addresses even if it
was).
With FEAT_TRF, the current behavior of trace in guests exists depends on
whether nVHE or VHE are being used. Both of the examples below are from
the host's point of view, as Coresight isn't accessible from guests.
This patchset is only relevant to when FEAT_TRF exists, otherwise there
is no change.
nVHE:
Because the host and the guest are both using TRFCR_EL1, trace will be
generated in guests depending on the same filter rules the host is
using. For example if the host is tracing userspace only, then guest
userspace trace will also be collected.
(This is further limited by whether TRBE is used because an issue
with TRBE means that it's completely disabled in nVHE guests, but it's
possible to have other tracing components.)
VHE:
With VHE, the host filters will be in TRFCR_EL2, but the filters in
TRFCR_EL1 will be active when the guest is running. Because we don't
write to TRFCR_EL1, guest trace will be completely disabled.
With this change, the guest filtering rules from the Perf session are
honored for both nVHE and VHE modes. This is done by either writing to
TRFCR_EL12 at the start of the Perf session and doing nothing else
further, or caching the guest value and writing it at guest switch for
nVHE.
The first commit moves the register to sysreg because I add the EL12
version.
---
Changes since V2:
* Add a new iflag to signify presence of FEAT_TRF and keep the
existing TRBE iflag. This fixes the issue where TRBLIMITR_EL1 was
being accessed even if TRBE didn't exist
* Reword a commit message
Changes since V1:
* Squashed all the arm64/tools/sysreg changes into the first commit
* Add a new commit to move SPE and TRBE regs into the kvm sysreg array
* Add a comment above the TRFCR global that it's per host CPU rather
than vcpu
Changes since nVHE RFC [1]:
* Re-write just in terms of the register value to be written for the
host and the guest. This removes some logic from the hyp code and
a value of kvm_vcpu_arch:trfcr_el1 = 0 no longer means "don't
restore".
* Remove all the conditional compilation and new files.
* Change the kvm_etm_update_vcpu_events macro to a function.
* Re-use DEBUG_STATE_SAVE_TRFCR so iflags don't need to be expanded
anymore.
* Expand the cover letter.
Changes since VHE v3 [2]:
* Use the same interface as nVHE mode so TRFCR_EL12 is now written by
kvm.
[1]: https://lore.kernel.org/kvmarm/20230804101317.460697-1-james.clark@arm.com/
[2]: https://lore.kernel.org/kvmarm/20230905102117.2011094-1-james.clark@arm.com/
James Clark (6):
arm64/sysreg: Move TRFCR definitions to sysreg
arm64: KVM: Move SPE and trace registers to the sysreg array
arm64: KVM: Add iflag for FEAT_TRF
arm64: KVM: Add interface to set guest value for TRFCR register
arm64: KVM: Write TRFCR value on guest switch with nVHE
coresight: Pass guest TRFCR value to KVM
arch/arm64/include/asm/kvm_host.h | 13 +--
arch/arm64/include/asm/kvm_hyp.h | 6 +-
arch/arm64/include/asm/sysreg.h | 12 ---
arch/arm64/kvm/arm.c | 1 +
arch/arm64/kvm/debug.c | 48 +++++++++-
arch/arm64/kvm/hyp/nvhe/debug-sr.c | 88 +++++++++++--------
arch/arm64/kvm/hyp/nvhe/switch.c | 4 +-
arch/arm64/tools/sysreg | 41 +++++++++
.../coresight/coresight-etm4x-core.c | 42 +++++++--
drivers/hwtracing/coresight/coresight-etm4x.h | 2 +-
drivers/hwtracing/coresight/coresight-priv.h | 3 +
11 files changed, 192 insertions(+), 68 deletions(-)
--
2.34.1
On Tue, Nov 07, 2023 at 12:16:25PM +0200, Adrian Hunter wrote:
[...]
> >>>> "If users know" <- how would users know? Could the kernel
> >>>> or tools also figure it out?
> >>>
> >>> Adrian, I'm trying to go all the outstanding patches, do you still have
> >>> any issues with this series?
> >>
> >> No, although the question wasn't actually answered. I presume users
> >> just have to try the 'T' option and see if it helps.
> >
> > Sometimes, users are software developers in SoC companies, they can
> > know well for the hardware design but are confused why current
> > implementation cannot use timestamp trace. This is the main reason
> > I sent this patch set.
> >
> > An example hardware platform is DB410c [1], we know its CoreSight can
> > support timestamp trace, but if without this adding option 'T', we
> > have no chance to use it due to it its CPU arch is prior to Armv8.4.
>
> perf config might be better than an itrace option, but you decide.
I understand perf config is a better approach due to users don't need
to bother inputting options after set it once. I will look at it and
respin new patch set.
Thanks for suggestion!
Leo