Differences

This shows you the differences between two versions of the page.

Link to this comparison view

--- shell [2016/07/21 08:11]
dblume [Application Memory Usage]
+++ shell [2019/01/17 21:25] (current)
dblume [jq Tips]
@@ Line 113: / Line 113: @@
 ((VERBOSE==1)) && echo "Starting at $(date)"
 rsync /cygdrive/c/Users/me /cygdrive/"$DISK"/backup/Users
+# We add "|| true" because we don't want to stop
+# if the directory was already empty
+rm -r /cygdrive/c/Users/me/tmp/* || true
+# Note how we find the number of cores to use
+make -C build_subdirectory all -j$(grep -c ^processor /proc/cpuinfo)
 </code>
@@ Line 380: / Line 387: @@
 done
 </code>
+Alternatively, to see the RSS use of that process alone:
+<code bash>
+while true; do sync; cat /proc/$(pidof yourprocess)/status | grep VmRSS; sleep 1; done
+</code>
+====== Measuring Available Memory ======
+This note doesn't entirely make sense to me. Maybe need to study up on "cat /proc/meminfo" vs. "cat /proc/vmstat" vs. "vmstat".
+The best measure I've found for "available memory" is nr_inactive_file_pages+nr_active_file_pages+nr_free_pages from /proc/vmstat. And then you have to subtract out some heuristically determined value which is base system working set. (That heuristically determined value can be 30-40MB.)
+The command ''free'' just isn't a great indicator in general of how much memory is available because it doesn't account for the cached  file-backed pages that could be dumped to make more memory available.
 ====== Shared Memory Usage ======
@@ Line 396: / Line 413: @@
 kernel.shmall= 268435456
 </code>
+====== Performance Metrics ======
+  * Use [[http://man7.org/linux/man-pages/man1/perf-timechart.1.html|perf-timechart]]
+  * [[https://github.com/gperftools/gperftools|gperftools]]
+And you can scrape logs that start with timecodes to create Spreadsheet charts. Given logs like:
+<code>
+-10-13 19:54:44  memory 22a4
+</code>
+On a Macintosh:
+<code bash>
+grep memory devicelogs.txt | tr -s ' ' | cut -d " " -f 1,2,4 | \
+sed 's/\([0-9\-]\+\) \([0-9:]\+\).[0-9]\+ \([0-9a-f]\+\)/\1,\2,=DATEVALUE("\1")+TIMEVALUE("\2"),=HEX2DEC("\3")/' >  heapinfo.csv; \
+open heapinfo.csv -a "Microsoft Excel"
+</code>
+And on Linux, instead of opening Microsoft Excel, that last line would be:
+<code bash>
+libreoffice --calc heapinfo.csv
+</code>
+====== Cron ======
+Keep tasks serialized with [[https://linux.die.net/man/1/flock|flock(1)]]:
+    (
+         flock -n 9 || exit 1
+         # ... commands executed under lock ...
+    ) 9>/var/lock/mylockfile
+====== Retrieving Symbols with addr2line ======
+You can gather a backtrace (stacktrace) with this piped command to addr2line.
+    $ cat << EOF | cut -d " " -f 3 | tr -d "[]" | \
+      addr2line -e builds/austin/src/platform/gibbon/netflix | \
+      xargs -d '\n' realpath --relative-to=.
+    > 7/22 app() [0xf7878] (0xf7878)
+    > 8/22 app() [0x39c2f8] (0x39c2f8)
+    > 9/22 app() [0xe1964] (0xe1964)
+    > EOF
+    src/Application.h:106 (discriminator 3)
+    src/platform/main.cpp:521
+    src/Application.cpp:95
+====== Sort by Frequency ======
+I ran the following P4 command to find out who's been editing a file recently:
+   $ find . -name fname.cpp | xargs p4 filelog -s -m 10 | grep -P "^\.\.\. #" | \
+     awk '{print $9}' | cut -d @ -f 1 | sort | uniq -c | sort -nr
+====== jq Tips ======
+jq is really handy. Here's a tip for some processing I often do:
+<file javascript fruits.txt>
+{ "fruits":
+  {
+    "apple":
+       {
+          "name": "Apple",
+          "price" : 2
+       },
+    "banana":
+       {
+          "name": "Banana",
+          "price" : 3
+       },
+    "count": 2,
+    "open": true
+  }
+}
+</file>
+<code>
+$ jq '.fruits|del(.count,.open)|with_entries(.value |= .price)' fruits.txt
+{
+  "apple": 2,
+  "banana": 3
+}
+# with_entries(f) is an alias for to_entries | map(x) | from_entries
+jq '.fruits|del(.count,.open)|to_entries|map(.value |= .price)|from_entries' f
+{
+  "apple": 2,
+  "banana": 3
+}
+$ jq '.fruits|del(.count,.open)|[to_entries[]|{(.key): .value.price}]|add' fruits.txt
+{
+  "apple": 2,
+  "banana": 3
+}
+</code>
+<file javascript fruit_ip.txt>
+{
+  "192.168.144.52": {
+    "ipAddress": "192.168.144.52",
+    "attributes": {
+      "model": "apple",
+      "name": "David's apple"
+    }
+  },
+  "192.168.144.40": {
+    "ipAddress": "192.168.144.40",
+    "attributes": {
+      "model": "banana",
+      "name": "David's banana"
+    }
+  }
+}
+</file>
+<code>
+$ jq '[to_entries[]|{"key":.value.attributes.name,"value":.key}]|from_entries' fruit_ip.txt
+{
+  "David's apple": "192.168.144.52",
+  "David's banana": "192.168.144.40"
+}
+$ jq '[.[]|{(.attributes.name):.ipAddress}]|add' fruit_ip.txt
+{
+  "David's apple": "192.168.144.52",
+  "David's banana": "192.168.144.40"
+}
+$ jq -r "to_entries|map(\"\(.value.attributes.name) = \(.key)\")|.[]" fruit_ip.txt
+David's apple = 192.168.144.52
+David's banana = 192.168.144.40
+</code>
 ====== Additional Keywords ======
 Linux, Unix, *nix