perf tools: Add example call-graph script
authorAdrian Hunter <adrian.hunter@intel.com>
Fri, 17 Jul 2015 16:33:45 +0000 (19:33 +0300)
committerArnaldo Carvalho de Melo <acme@redhat.com>
Fri, 21 Aug 2015 15:32:40 +0000 (12:32 -0300)
Add a script to produce a call-graph from data exported to a postgresql
database and derived from a processor trace event like intel_pt or intel_bts.

Refer to comments in the scripts call-graph-from-postgresql.py and
export-to-postgresql.py for more details on how to set up the environment,
install the required packages, etc.

Committer note:

From the scripts, for convenience while reading 'git log':

  An example of using this script with Intel PT:

  $ perf record -e intel_pt//u ls
  $ perf script -s ~/libexec/perf-core/scripts/python/export-to-postgresql.py pt_example branches calls
  2015-05-29 12:49:23.464364 Creating database...
  2015-05-29 12:49:26.281717 Writing to intermediate files...
  2015-05-29 12:49:27.190383 Copying to database...
  2015-05-29 12:49:28.140451 Removing intermediate files...
  2015-05-29 12:49:28.147451 Adding primary keys
  2015-05-29 12:49:28.655683 Adding foreign keys
  2015-05-29 12:49:29.365350 Done
  $ python tools/perf/scripts/python/call-graph-from-postgresql.py pt_example
  # The result is a GUI window with a tree representing a context-sensitive
  # call-graph.  Expanding a couple of levels of the tree and adjusting column
  # widths to suit will display something like:

                                         Call Graph: pt_example
  Call Path                        |Object     |Count|Time(ns)|Time(%)|Branch Count|Branch Count(%)
  v- ls
     v- 2638:2638
         v- _start                  ld-2.19.so    1   10074071  100.0        211135          100.0
           |- unknown               unknown       1      13198    0.1             1            0.0
           >- _dl_start             ld-2.19.so    1    1400980   13.9         19637            9.3
           >- _d_linit_internal     ld-2.19.so    1     448152    4.4         11094            5.3
           v-__libc_start_main@plt  ls            1    8211741   81.5        180397           85.4
              >- _dl_fixup          ld-2.19.so    1       7607    0.1           108            0.1
              >- __cxa_atexit       libc-2.19.so  1      11737    0.1            10            0.0
              >- __libc_csu_init    ls            1      10354    0.1            10            0.0
              |- _setjmp            libc-2.19.so  1          0    0.0             4            0.0
              v- main               ls            1    8182043   99.6        180254           99.9

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/1437150840-31811-11-git-send-email-adrian.hunter@intel.com
[ Added 'python-pyside qt-postgresql' to the yum cmdline installing required packages ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
tools/perf/scripts/python/call-graph-from-postgresql.py [new file with mode: 0644]
tools/perf/scripts/python/export-to-postgresql.py

diff --git a/tools/perf/scripts/python/call-graph-from-postgresql.py b/tools/perf/scripts/python/call-graph-from-postgresql.py
new file mode 100644 (file)
index 0000000..e78fdc2
--- /dev/null
@@ -0,0 +1,327 @@
+#!/usr/bin/python2
+# call-graph-from-postgresql.py: create call-graph from postgresql database
+# Copyright (c) 2014, Intel Corporation.
+#
+# This program is free software; you can redistribute it and/or modify it
+# under the terms and conditions of the GNU General Public License,
+# version 2, as published by the Free Software Foundation.
+#
+# This program is distributed in the hope it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+# more details.
+
+# To use this script you will need to have exported data using the
+# export-to-postgresql.py script.  Refer to that script for details.
+#
+# Following on from the example in the export-to-postgresql.py script, a
+# call-graph can be displayed for the pt_example database like this:
+#
+#      python tools/perf/scripts/python/call-graph-from-postgresql.py pt_example
+#
+# Note this script supports connecting to remote databases by setting hostname,
+# port, username, password, and dbname e.g.
+#
+#      python tools/perf/scripts/python/call-graph-from-postgresql.py "hostname=myhost username=myuser password=mypassword dbname=pt_example"
+#
+# The result is a GUI window with a tree representing a context-sensitive
+# call-graph.  Expanding a couple of levels of the tree and adjusting column
+# widths to suit will display something like:
+#
+#                                         Call Graph: pt_example
+# Call Path                          Object      Count   Time(ns)  Time(%)  Branch Count   Branch Count(%)
+# v- ls
+#     v- 2638:2638
+#         v- _start                  ld-2.19.so    1     10074071   100.0         211135            100.0
+#           |- unknown               unknown       1        13198     0.1              1              0.0
+#           >- _dl_start             ld-2.19.so    1      1400980    13.9          19637              9.3
+#           >- _d_linit_internal     ld-2.19.so    1       448152     4.4          11094              5.3
+#           v-__libc_start_main@plt  ls            1      8211741    81.5         180397             85.4
+#              >- _dl_fixup          ld-2.19.so    1         7607     0.1            108              0.1
+#              >- __cxa_atexit       libc-2.19.so  1        11737     0.1             10              0.0
+#              >- __libc_csu_init    ls            1        10354     0.1             10              0.0
+#              |- _setjmp            libc-2.19.so  1            0     0.0              4              0.0
+#              v- main               ls            1      8182043    99.6         180254             99.9
+#
+# Points to note:
+#      The top level is a command name (comm)
+#      The next level is a thread (pid:tid)
+#      Subsequent levels are functions
+#      'Count' is the number of calls
+#      'Time' is the elapsed time until the function returns
+#      Percentages are relative to the level above
+#      'Branch Count' is the total number of branches for that function and all
+#       functions that it calls
+
+import sys
+from PySide.QtCore import *
+from PySide.QtGui import *
+from PySide.QtSql import *
+from decimal import *
+
+class TreeItem():
+
+       def __init__(self, db, row, parent_item):
+               self.db = db
+               self.row = row
+               self.parent_item = parent_item
+               self.query_done = False;
+               self.child_count = 0
+               self.child_items = []
+               self.data = ["", "", "", "", "", "", ""]
+               self.comm_id = 0
+               self.thread_id = 0
+               self.call_path_id = 1
+               self.branch_count = 0
+               self.time = 0
+               if not parent_item:
+                       self.setUpRoot()
+
+       def setUpRoot(self):
+               self.query_done = True
+               query = QSqlQuery(self.db)
+               ret = query.exec_('SELECT id, comm FROM comms')
+               if not ret:
+                       raise Exception("Query failed: " + query.lastError().text())
+               while query.next():
+                       if not query.value(0):
+                               continue
+                       child_item = TreeItem(self.db, self.child_count, self)
+                       self.child_items.append(child_item)
+                       self.child_count += 1
+                       child_item.setUpLevel1(query.value(0), query.value(1))
+
+       def setUpLevel1(self, comm_id, comm):
+               self.query_done = True;
+               self.comm_id = comm_id
+               self.data[0] = comm
+               self.child_items = []
+               self.child_count = 0
+               query = QSqlQuery(self.db)
+               ret = query.exec_('SELECT thread_id, ( SELECT pid FROM threads WHERE id = thread_id ), ( SELECT tid FROM threads WHERE id = thread_id ) FROM comm_threads WHERE comm_id = ' + str(comm_id))
+               if not ret:
+                       raise Exception("Query failed: " + query.lastError().text())
+               while query.next():
+                       child_item = TreeItem(self.db, self.child_count, self)
+                       self.child_items.append(child_item)
+                       self.child_count += 1
+                       child_item.setUpLevel2(comm_id, query.value(0), query.value(1), query.value(2))
+
+       def setUpLevel2(self, comm_id, thread_id, pid, tid):
+               self.comm_id = comm_id
+               self.thread_id = thread_id
+               self.data[0] = str(pid) + ":" + str(tid)
+
+       def getChildItem(self, row):
+               return self.child_items[row]
+
+       def getParentItem(self):
+               return self.parent_item
+
+       def getRow(self):
+               return self.row
+
+       def timePercent(self, b):
+               if not self.time:
+                       return "0.0"
+               x = (b * Decimal(100)) / self.time
+               return str(x.quantize(Decimal('.1'), rounding=ROUND_HALF_UP))
+
+       def branchPercent(self, b):
+               if not self.branch_count:
+                       return "0.0"
+               x = (b * Decimal(100)) / self.branch_count
+               return str(x.quantize(Decimal('.1'), rounding=ROUND_HALF_UP))
+
+       def addChild(self, call_path_id, name, dso, count, time, branch_count):
+               child_item = TreeItem(self.db, self.child_count, self)
+               child_item.comm_id = self.comm_id
+               child_item.thread_id = self.thread_id
+               child_item.call_path_id = call_path_id
+               child_item.branch_count = branch_count
+               child_item.time = time
+               child_item.data[0] = name
+               if dso == "[kernel.kallsyms]":
+                       dso = "[kernel]"
+               child_item.data[1] = dso
+               child_item.data[2] = str(count)
+               child_item.data[3] = str(time)
+               child_item.data[4] = self.timePercent(time)
+               child_item.data[5] = str(branch_count)
+               child_item.data[6] = self.branchPercent(branch_count)
+               self.child_items.append(child_item)
+               self.child_count += 1
+
+       def selectCalls(self):
+               self.query_done = True;
+               query = QSqlQuery(self.db)
+               ret = query.exec_('SELECT id, call_path_id, branch_count, call_time, return_time, '
+                                 '( SELECT name FROM symbols WHERE id = ( SELECT symbol_id FROM call_paths WHERE id = call_path_id ) ), '
+                                 '( SELECT short_name FROM dsos WHERE id = ( SELECT dso_id FROM symbols WHERE id = ( SELECT symbol_id FROM call_paths WHERE id = call_path_id ) ) ), '
+                                 '( SELECT ip FROM call_paths where id = call_path_id ) '
+                                 'FROM calls WHERE parent_call_path_id = ' + str(self.call_path_id) + ' AND comm_id = ' + str(self.comm_id) + ' AND thread_id = ' + str(self.thread_id) +
+                                 'ORDER BY call_path_id')
+               if not ret:
+                       raise Exception("Query failed: " + query.lastError().text())
+               last_call_path_id = 0
+               name = ""
+               dso = ""
+               count = 0
+               branch_count = 0
+               total_branch_count = 0
+               time = 0
+               total_time = 0
+               while query.next():
+                       if query.value(1) == last_call_path_id:
+                               count += 1
+                               branch_count += query.value(2)
+                               time += query.value(4) - query.value(3)
+                       else:
+                               if count:
+                                       self.addChild(last_call_path_id, name, dso, count, time, branch_count)
+                               last_call_path_id = query.value(1)
+                               name = query.value(5)
+                               dso = query.value(6)
+                               count = 1
+                               total_branch_count += branch_count
+                               total_time += time
+                               branch_count = query.value(2)
+                               time = query.value(4) - query.value(3)
+               if count:
+                       self.addChild(last_call_path_id, name, dso, count, time, branch_count)
+               total_branch_count += branch_count
+               total_time += time
+               # Top level does not have time or branch count, so fix that here
+               if total_branch_count > self.branch_count:
+                       self.branch_count = total_branch_count
+                       if self.branch_count:
+                               for child_item in self.child_items:
+                                       child_item.data[6] = self.branchPercent(child_item.branch_count)
+               if total_time > self.time:
+                       self.time = total_time
+                       if self.time:
+                               for child_item in self.child_items:
+                                       child_item.data[4] = self.timePercent(child_item.time)
+
+       def childCount(self):
+               if not self.query_done:
+                       self.selectCalls()
+               return self.child_count
+
+       def columnCount(self):
+               return 7
+
+       def columnHeader(self, column):
+               headers = ["Call Path", "Object", "Count ", "Time (ns) ", "Time (%) ", "Branch Count ", "Branch Count (%) "]
+               return headers[column]
+
+       def getData(self, column):
+               return self.data[column]
+
+class TreeModel(QAbstractItemModel):
+
+       def __init__(self, db, parent=None):
+               super(TreeModel, self).__init__(parent)
+               self.db = db
+               self.root = TreeItem(db, 0, None)
+
+       def columnCount(self, parent):
+               return self.root.columnCount()
+
+       def rowCount(self, parent):
+               if parent.isValid():
+                       parent_item = parent.internalPointer()
+               else:
+                       parent_item = self.root
+               return parent_item.childCount()
+
+       def headerData(self, section, orientation, role):
+               if role == Qt.TextAlignmentRole:
+                       if section > 1:
+                               return Qt.AlignRight
+               if role != Qt.DisplayRole:
+                       return None
+               if orientation != Qt.Horizontal:
+                       return None
+               return self.root.columnHeader(section)
+
+       def parent(self, child):
+               child_item = child.internalPointer()
+               if child_item is self.root:
+                       return QModelIndex()
+               parent_item = child_item.getParentItem()
+               return self.createIndex(parent_item.getRow(), 0, parent_item)
+
+       def index(self, row, column, parent):
+               if parent.isValid():
+                       parent_item = parent.internalPointer()
+               else:
+                       parent_item = self.root
+               child_item = parent_item.getChildItem(row)
+               return self.createIndex(row, column, child_item)
+
+       def data(self, index, role):
+               if role == Qt.TextAlignmentRole:
+                       if index.column() > 1:
+                               return Qt.AlignRight
+               if role != Qt.DisplayRole:
+                       return None
+               index_item = index.internalPointer()
+               return index_item.getData(index.column())
+
+class MainWindow(QMainWindow):
+
+       def __init__(self, db, dbname, parent=None):
+               super(MainWindow, self).__init__(parent)
+
+               self.setObjectName("MainWindow")
+               self.setWindowTitle("Call Graph: " + dbname)
+               self.move(100, 100)
+               self.resize(800, 600)
+               style = self.style()
+               icon = style.standardIcon(QStyle.SP_MessageBoxInformation)
+               self.setWindowIcon(icon);
+
+               self.model = TreeModel(db)
+
+               self.view = QTreeView()
+               self.view.setModel(self.model)
+
+               self.setCentralWidget(self.view)
+
+if __name__ == '__main__':
+       if (len(sys.argv) < 2):
+               print >> sys.stderr, "Usage is: call-graph-from-postgresql.py <database name>"
+               raise Exception("Too few arguments")
+
+       dbname = sys.argv[1]
+
+       db = QSqlDatabase.addDatabase('QPSQL')
+
+       opts = dbname.split()
+       for opt in opts:
+               if '=' in opt:
+                       opt = opt.split('=')
+                       if opt[0] == 'hostname':
+                               db.setHostName(opt[1])
+                       elif opt[0] == 'port':
+                               db.setPort(int(opt[1]))
+                       elif opt[0] == 'username':
+                               db.setUserName(opt[1])
+                       elif opt[0] == 'password':
+                               db.setPassword(opt[1])
+                       elif opt[0] == 'dbname':
+                               dbname = opt[1]
+               else:
+                       dbname = opt
+
+       db.setDatabaseName(dbname)
+       if not db.open():
+               raise Exception("Failed to open database " + dbname + " error: " + db.lastError().text())
+
+       app = QApplication(sys.argv)
+       window = MainWindow(db, dbname)
+       window.show()
+       err = app.exec_()
+       db.close()
+       sys.exit(err)
index 4cdafd880074c6882a8ef558672cde6079eac8e7..84a32037a80fff854108b21cc66eb5ebadab2d21 100644 (file)
@@ -15,6 +15,53 @@ import sys
 import struct
 import datetime
 
+# To use this script you will need to have installed package python-pyside which
+# provides LGPL-licensed Python bindings for Qt.  You will also need the package
+# libqt4-sql-psql for Qt postgresql support.
+#
+# The script assumes postgresql is running on the local machine and that the
+# user has postgresql permissions to create databases. Examples of installing
+# postgresql and adding such a user are:
+#
+# fedora:
+#
+#      $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql
+#      $ sudo su - postgres -c initdb
+#      $ sudo service postgresql start
+#      $ sudo su - postgres
+#      $ createuser <your user id here>
+#      Shall the new role be a superuser? (y/n) y
+#
+# ubuntu:
+#
+#      $ sudo apt-get install postgresql
+#      $ sudo su - postgres
+#      $ createuser <your user id here>
+#      Shall the new role be a superuser? (y/n) y
+#
+# An example of using this script with Intel PT:
+#
+#      $ perf record -e intel_pt//u ls
+#      $ perf script -s ~/libexec/perf-core/scripts/python/export-to-postgresql.py pt_example branches calls
+#      2015-05-29 12:49:23.464364 Creating database...
+#      2015-05-29 12:49:26.281717 Writing to intermediate files...
+#      2015-05-29 12:49:27.190383 Copying to database...
+#      2015-05-29 12:49:28.140451 Removing intermediate files...
+#      2015-05-29 12:49:28.147451 Adding primary keys
+#      2015-05-29 12:49:28.655683 Adding foreign keys
+#      2015-05-29 12:49:29.365350 Done
+#
+# To browse the database, psql can be used e.g.
+#
+#      $ psql pt_example
+#      pt_example=# select * from samples_view where id < 100;
+#      pt_example=# \d+
+#      pt_example=# \d+ samples_view
+#      pt_example=# \q
+#
+# An example of using the database is provided by the script
+# call-graph-from-postgresql.py.  Refer to that script for details.
+
 from PySide.QtSql import *
 
 # Need to access PostgreSQL C library directly to use COPY FROM STDIN