# Memory usage

This file contains information about memory management and diagnosis in Zircon,
and talks about ways to examine process and system memory usage.

A process can use memory 3 ways:

 1. Mapped memory in the form of heaps, thread stacks, executable code + data.
    This memory is represented by [VMARs](/docs/reference/kernel_objects/vm_address_region.md)
    which in turn hold a reference to [VMOs](/docs/reference/kernel_objects/vm_object.md).
    The programmer usually interfaces with this memory via memory addresses.
 2. Stand-alone VMOs. These are sets of memory pages that are not mapped via a
    VMAR. The programmer interfaces with this memory via handles; usually issuing
    [vmo_read](/docs/reference/syscalls/vmo_read.md) and [vmo_write](/docs/reference/syscalls/vmo_write.md).
 3. Kernel memory in the form of handles to kernel objects.

Fuchsia follows an over-commit model: processes can allocate more memory than
can be satisfied at a given moment and as memory pages are written they are
physically allocated (wired) on the fly by kernel.

## Userspace memory

Which processes are using all of the memory?

### Dump total process memory usage

Use the `ps` tool:

```
$ ps
TASK           PSS PRIVATE  SHARED NAME
j:1028       32.9M   32.8M         root
  p:1043   1386.3k   1384k     28k bin/devmgr
  j:1082     30.0M   30.0M         zircon-drivers
    p:1209  774.3k    772k     28k /boot/bin/acpisvc
    p:1565  250.3k    248k     28k driver_host:root
    p:1619  654.3k    652k     28k driver_host:misc
    p:1688  258.3k    256k     28k driver_host:platform
    p:1867 3878.3k   3876k     28k driver_host:pci#1:1234:1111
    p:1916   24.4M   24.4M     28k driver_host:pci#3:8086:2922
  j:1103   1475.7k   1464k         zircon-services
    p:1104  298.3k    296k     28k crashlogger
    p:1290  242.3k    240k     28k netsvc
    p:2115  362.3k    360k     28k sh:console
    p:2334  266.3k    264k     28k sh:vc
    p:2441  306.3k    304k     28k /boot/bin/ps
TASK           PSS PRIVATE  SHARED NAME
```

**PSS** (proportional shared state) is a number of bytes that estimates how much
in-process mapped physical memory the process consumes. Its value is `PRIVATE +
(SHARED / sharing-ratio)`, where `sharing-ratio` is based on the number of
processes that share each of the pages in this process.

The intent is that, e.g., if four processes share a single page, 1/4 of the
bytes of that page is included in each of the four process's `PSS`. If two
processes share a different page, then each gets 1/2 of that page's bytes.

**PRIVATE** is the number of bytes that are mapped only by this process. I.e.,
no other process maps this memory. Note that this does not account for private
VMOs that are not mapped.

**SHARED** is the number of bytes that are mapped by this process and at least
one other process. Note that this does not account for shared VMOs that are not
mapped. It also does not indicate how many processes share the memory: it could
be 2, it could be 50.

### Visualize memory usage

If you have a Fuchsia build, you can use treemap to visualize memory usage by
the system.

 1. On your host machine, run the following command from the root of your
    Fuchsia checkout:

    ```./scripts/fx shell memgraph -vt | ./scripts/memory/treemap.py > mem.html```

 2. Open `mem.html` in a browser.

The `memgraph` tool generates a JSON description of system task and memory
information, which is then parsed by the `treemap.py` script. `-vt` says
to include VMOs and threads in the output.

### Dump a process's detailed memory maps

If you want to see why a specific process uses so much memory, you can run the
`vmaps` tool on its koid (koid is the ID that shows up when running ps) to see
what it has mapped into memory.

```
$ vmaps help
Usage: vmaps <process-koid>

Dumps a process's memory maps to stdout.

First column:
  "/A" -- Process address space
  "/R" -- Root VMAR
  "R"  -- VMAR (R for Region)
  "M"  -- Mapping

  Indentation indicates parent/child relationship.
```

Column tags:

-   `:sz`: The virtual size of the entry, in bytes. Not all pages are
    necessarily backed by physical memory.
-   `:res`: The amount of memory "resident" in the entry, in bytes; i.e., the
    amount of physical memory that backs the entry. This memory may be private
    (only accessible by this process) or shared by multiple processes.
-   `:vmo`: The `koid` of the VMO mapped into this region.

```
$ vmaps 2470
/A ________01000000-00007ffffffff000    128.0T:sz                    'proc:2470'
/R ________01000000-00007ffffffff000    128.0T:sz                    'root'
...
# This 'R' region is a dynamic library. The r-x section is .text, the r--
# section is .rodata, and the rw- section is .data + .bss.
R  00000187bc867000-00000187bc881000      104k:sz                    'useralloc'
 M 00000187bc867000-00000187bc87d000 r-x   88k:sz   0B:res  2535:vmo 'libfdio.so'
 M 00000187bc87e000-00000187bc87f000 r--    4k:sz   4k:res  2537:vmo 'libfdio.so'
 M 00000187bc87f000-00000187bc881000 rw-    8k:sz   8k:res  2537:vmo 'libfdio.so'
...
# This 2MB anonymous mapping is probably part of the heap.
M  0000246812b91000-0000246812d91000 rw-    2M:sz  76k:res  2542:vmo 'mmap-anonymous'
...
# This region looks like a stack: a big chunk of virtual space (:sz) with a
# slightly-smaller mapping inside (accounting for a 4k guard page), and only a
# small amount actually committed (:res).
R  0000358923d92000-0000358923dd3000      260k:sz                    'useralloc'
 M 0000358923d93000-0000358923dd3000 rw-  256k:sz  16k:res  2538:vmo ''
...
# The stack for the initial thread, which is allocated differently.
M  0000400cbba84000-0000400cbbac4000 rw-  256k:sz   4k:res  2513:vmo 'initial-stack'
...
# The vDSO, which only has .text and .rodata.
R  000047e1ab874000-000047e1ab87b000       28k:sz                    'useralloc'
 M 000047e1ab874000-000047e1ab87a000 r--   24k:sz  24k:res  1031:vmo 'vdso/full'
 M 000047e1ab87a000-000047e1ab87b000 r-x    4k:sz   4k:res  1031:vmo 'vdso/full'
...
# The main binary for this process.
R  000059f5c7068000-000059f5c708d000      148k:sz                    'useralloc'
 M 000059f5c7068000-000059f5c7088000 r-x  128k:sz   0B:res  2476:vmo '/boot/bin/sh'
 M 000059f5c7089000-000059f5c708b000 r--    8k:sz   8k:res  2517:vmo '/boot/bin/sh'
 M 000059f5c708b000-000059f5c708d000 rw-    8k:sz   8k:res  2517:vmo '/boot/bin/sh'
...
```

### Dump all VMOs associated with a process

```
vmos <pid>
```

This will also show unmapped VMOs, which neither `ps` nor `vmaps` currently
account for.

It also shows whether a given VMO is a child, along with its parent's koid.

```
$ vmos 1118
rights  koid parent #chld #map #shr    size   alloc name
rwxmdt  1170      -     0    1    1      4k      4k stack: msg of 0x5a
r-xmdt  1031      -     2   28   14     28k     28k vdso/full
     -  1298      -     0    1    1      2M     68k jemalloc-heap
     -  1381      -     0    3    1    516k      8k self-dump-thread:0x12afe79c8b38
     -  1233   1232     1    1    1   33.6k      4k libbacktrace.so
     -  1237   1233     0    1    1      4k      4k data:libbacktrace.so
...
     -  1153   1146     1    1    1  883.2k     12k ld.so.1
     -  1158   1153     0    1    1     16k     12k data:ld.so.1
     -  1159      -     0    1    1     12k     12k bss:ld.so.1
rights  koid parent #chld #map #shr    size   alloc name
```

Columns:

-   `rights`: If the process points to the VMO via a handle, this column shows
    the rights that the handle has, zero or more of:
    -   `r`: `ZX_RIGHT_READ`
    -   `w`: `ZX_RIGHT_WRITE`
    -   `x`: `ZX_RIGHT_EXECUTE`
    -   `m`: `ZX_RIGHT_MAP`
    -   `d`: `ZX_RIGHT_DUPLICATE`
    -   `t`: `ZX_RIGHT_TRANSFER`
    -   **NOTE**: Non-handle entries will have a single '-' in this column.
-   `koid`: The koid of the VMO, if it has one. Zero otherwise. A VMO without a
    koid was created by the kernel, and has never had a userspace handle.
-   `parent`: The koid of the VMO's parent, if it's a child.
-   `#chld`: The number of active children of the VMO.
-   `#map`: The number of times the VMO is currently mapped into VMARs.
-   `#shr`: The number of processes that map (share) the VMO.
-   `size`: The VMO's current size, in bytes.
-   `alloc`: The amount of physical memory allocated to the VMO, in bytes.
    -   **NOTE**: If this column contains the value `phys`, it means that the
        VMO points to a raw physical address range like a memory-mapped device.
        `phys` VMOs do not consume RAM.
-   `name`: The name of the VMO, or `-` if its name is empty.

To relate this back to `ps`: each VMO contributes, for its mapped portions
(since not all or any of a VMO's pages may be mapped):

```
PRIVATE =  #shr == 1 ? alloc : 0
SHARED  =  #shr  > 1 ? alloc : 0
PSS     =  PRIVATE + (SHARED / #shr)
```

### Dump "hidden" (unmapped and kernel) VMOs

Note: This is a kernel command, and will print to the kernel console.

```
k zx vmos hidden
```

Similar to `vmos <pid>`, but dumps all VMOs in the system that are not mapped
into any process:

-   VMOs that userspace has handles to but does not map
-   VMOs that are mapped only into kernel space
-   Kernel-only, unmapped VMOs that have no handles

A `koid` value of zero means that only the kernel has a reference to that VMO.

A `#map` value of zero means that the VMO is not mapped into any address space.

**See also**: `k zx vmos all`, which dumps all VMOs in the system. **NOTE**:
It's very common for this output to be truncated because of kernel console
buffer limitations, so it's often better to combine the `k zx vmos hidden`
output with a `vmaps` dump of each user process.

### Limitations

Neither `ps` nor `vmaps` currently account for:

-   VMOs or VMO subranges that are not mapped. E.g., you could create a VMO,
    write 1G of data into it, and it won't show up here.

None of the process-dumping tools account for:

-   Multiply-mapped pages. If you create multiple mappings using the same range
    of a VMO, any committed pages of the VMO will be counted as many times as
    those pages are mapped. This could be inside the same process, or could be
    between processes if those processes share a VMO.

    Note that "multiply-mapped pages" includes copy-on-write.
-   Underlying kernel memory overhead for resources allocated by a process.
    E.g., a process could have a million handles open, and those handles consume
    kernel memory.

    You can look at process handle consumption with the `k zx ps` command; run
    `k zx ps help` for a description of its columns.
-   Copy-on-write (COW) cloned VMOs. The clean (non-dirty, non-copied) pages of
    a clone will not count towards "shared" for a process that maps the clone,
    and those same pages may mistakenly count towards "private" of a process
    that maps the parent (cloned) VMO.

    TODO(dbort): Fix this; the tools were written before COW clones existed.

## Kernel memory

### Dump system memory arenas and kernel heap usage

Running `kstats -m` will continuously dump information about physical memory
usage and availability.

```
$ kstats -m
--- 2017-06-07T05:51:08.021Z ---
mem total      free      VMOs     kheap     kfree     wired       mmu       ipc     other
    2048M   1686.4M    317.8M      5.1M      0.9M     17.8M     20.0M      0.1M      0.0M

--- 2017-06-07T05:51:09.021Z ---
...
```

Fields:

-   The `-t` option show the timestamp `2017-06-07T05:51:08.021Z`, when the stats
    were collected, as an ISO 8601 string.
-   `total`: The total amount of physical memory available to the system.
-   `free`: The amount of unallocated memory.
-   `VMOs`: The amount of memory committed to VMOs, both kernel and user. A
    superset of all userspace memory. Does not include certain VMOs that fall
    under `wired`.
-   `kheap`: The amount of kernel heap memory marked as allocated.
-   `kfree`: The amount of kernel heap memory marked as free.
-   `wired`: The amount of memory reserved by and mapped into the kernel for
    reasons not covered by other fields in this struct. Typically for readonly
    data like the ram disk and kernel image, and for early-boot dynamic memory.
-   `mmu`: The amount of memory used for architecture-specific MMU metadata like
    page tables.
-   `ipc`: The amount of memory used for interprocess communication.
-   `other`: Everything else as `other`.

### Dump the kernel address space

Note: This is a kernel command, and will print to the kernel console.

```
k zx asd kernel
```

Dumps the kernel's VMAR/mapping/VMO hierarchy, similar to the `vmaps` tool for
user processes.

```
$ k zx asd kernel
as 0xffffffff80252b20 [0xffffff8000000000 0xffffffffffffffff] sz 0x8000000000 fl 0x1 ref 71 'kernel'
  vmar 0xffffffff802529a0 [0xffffff8000000000 0xffffffffffffffff] sz 0x8000000000 ref 1 'root'
    map 0xffffff80015f89a0 [0xffffff8000000000 0xffffff8fffffffff] sz 0x1000000000 mmufl 0x18 vmo 0xffffff80015f8890/k0 off 0 p ages 0 ref 1 ''
      vmo 0xffffff80015f8890/k0 size 0 pages 0 ref 1 parent k0
    map 0xffffff80015f8b30 [0xffffff9000000000 0xffffff9000000fff] sz 0x1000 mmufl 0x18 vmo 0xffffff80015f8a40/k0 off 0 pages 0 ref 1 ''
      object 0xffffff80015f8a40 base 0x7ffe2000 size 0x1000 ref 1
    map 0xffffff80015f8cc0 [0xffffff9000001000 0xffffff9000001fff] sz 0x1000 mmufl 0x1a vmo 0xffffff80015f8bd0/k0 off 0 pages 0 ref 1 ''
      object 0xffffff80015f8bd0 base 0xfed00000 size 0x1000 ref 1
...
```