tp: expand sparse columns only where a step needs them **Why** A scan expanded every sparse-null column into a slot per row for each batch, before any filter after it dropped rows. A join that filters a table once per outer row then paid for laying out columns of rows it threw away, and some stdlib queries ran 20 times slower on the executor than on the interpreter. **Changes** - Scans lend sparse columns as they are stored, as a new `ColumnView::Kind::kSparse` view carrying the column's prefix popcount. The popcount is the column's own when it keeps one, and is otherwise built once per cursor. - A new `Expand` transform lays sparse columns out for the rows the batch keeps. - Lowering puts an `Expand` in front of the first step that reads each sparse column, in front of any operator, and before the output for the columns the caller reads. Columns that share a point share one `Expand`. - `ColumnView::data()` and `At()` DCHECK that they are not given a sparse view. **Testing** - `DataframeScanTest`, `SearchScansDiffTest` and `TreeNumberNodesTest` run an `Expand` after the scan, as lowering does. - `perfetto_unittests` and all diff tests pass.
Perfetto is an open-source suite of tools for recording, visualizing and analyzing traces: timelines of what software and the system under it were doing. It is the default tracing system for Android and Chromium. System-wide recording works on Android and Linux. The tracing SDK, the UI and Trace Processor also run on macOS and Windows.
perf, Firefox Profiler, macOS Instruments, pprof, systrace, Fuchsia and more.Open an example Android trace without installing anything, or drag your own trace file onto ui.perfetto.dev.
On Android, with a device connected over adb:
curl -O https://raw.githubusercontent.com/google/perfetto/main/tools/record_android_trace python3 record_android_trace -o trace.pftrace -t 10s sched freq gfx view am wm
The trace opens in the Perfetto UI when recording finishes.
On Linux, record scheduling activity with tracebox:
curl -LO https://get.perfetto.dev/tracebox && chmod +x tracebox curl -LO https://raw.githubusercontent.com/google/perfetto/main/test/configs/scheduling.cfg sudo ./tracebox -o trace.pftrace --txt -c scheduling.cfg
For Chrome, use the recording page in the UI. To trace your own app, see in-app tracing.
curl -LO https://get.perfetto.dev/trace_processor && chmod +x trace_processor ./trace_processor query trace.pftrace \ "SELECT name, COUNT(*) AS cnt FROM slice GROUP BY name ORDER BY cnt DESC LIMIT 10"
Or from Python:
pip install perfetto
from perfetto.trace_processor import TraceProcessor tp = TraceProcessor(trace='trace.pftrace') print(tp.query('SELECT ts, dur, name FROM slice LIMIT 10').as_pandas_dataframe())
ftrace. (Linux cookbook)chrome://tracing, for debugging the browser, V8 and Blink. (Chrome tracing)Not sure where to start? How do I start using Perfetto? picks a path based on what you want to do.
Perfetto is a collection of components that work together:
traced, traced_probes) collect data from many processes and the kernel into a single trace file.See the documentation for the full picture.
See Community and getting help for more. We follow Google's Open Source Community Guidelines.
Contributions are welcome. The contributor guide covers building, testing and sending pull requests.
The Debian packaging of Perfetto is maintained in the Debian Perfetto Salsa repository.
Apache 2.0. See LICENSE for details.