XenOprofile, OProfile on Ubuntu 10.04 - Unsuccessful (2011/01/11)

OProfile - a system-wide and process profiler

Profiling runs in the background and data are collected at any time. It makes use of the hardware performance counters.

opcontrol : starting and stopping the OProfile daemon and providing setup parameters. 

opreport : gives image and symbol-based profile summaries for the whole system  or a process

Installation

># sudo apt-get install oprofile

Check for profile support in the kernel

># cat /boot/config-`uname -r` | grep PROFILE

The following command tells OProfile that we don't have an uncompressed kernel and it assign all kernel samples to a black-box called "no-vmlinux". With the uncompressesd kernel, it's possible to get a breakdown of what is happening inside of your kernel in more detail. 

># opcontrol --no-vmlinux

--no-vmlinux : indicates that we are not interested in recording samples for the kernel.

Basic Usage

To clean previous samples

># opcontrol --reset

To configure and start OProfile

># opcontrol --setup --no-vmlinux --event=CPU_CLK_UNHALTED:6000

Measuring CPU_CLK_UNHALTED events with a sample recorded for every 6000 events.

># opcontrol --start

Run my benchmark

># ./cpu -a 15 -n 1

># opcontrol --shutdown

># opreport -lt1

-t1: sets a threshold at 1% so that we don't see every symbol.

-l: shows you side each process

What is the meaning of the samples here in the output??

List all probable events

># opcontrol --list-events

It's possible to sample more events at a time. I think...

># opcontrol --setup --no-vmlinux --event=CPU_CLK_UNHALTED:6000 --event=LLC_MISSES:6000

Using OProfile, can we distinguish whether the application is CPU, memory, disk, network-intensive?

What kinds of metrics should I use?

XenOProfile

We are require to build Xenoprof from source code with Xen patch. 

First we need to get the source. 

># wget http://prdownloads.sourceforge.net/oprofile/oprofile-0.9.5.tar.gz

># tar -xzvf oprofile-0.9.5.tar.gz; cd oprofile-0.9.5

># wget http://xenoprof.sourceforge.net/oprofile-0.9.5-xen.patch

># patch -p1 < oprofile-0.9.5-xen.patch

># ./configure --with-kernel-support

configure: error: liberty library not found

># apt-get install binutils-dev

># ./configure --with-kernel-support

># make

># make install

In case you are getting the following error, change the sh shell to bash. 

># opcontrol --help

/usr/local/bin/opcontrol: 567: Syntax error: Bad for loop variable

># vim /usr/local/bin/opcontrol

#!/bin/bash

#

# opcontrol is a script to control OProfile

# opcontrol --help and opcontrol --list-events have info

#

# Copyright 2002

# Read the file COPYING

It should work now. 

># opcontrol --help

Active vs. Passive Profiling

Xennoprofile supports both active and passive mode for domain profiling. In the passive mode, the results indicate which domain is running at sample time, but don't get much deeper information. It's useful to get a quick look at which domains are used in the system. 

In active mode, each domain U needs to have its own instance of OProfile running, but it provides much better granularity than passive mode. Only paravirtualized domains can run in active mode. 

For Xen build, I just followed the instruction at this site. 

http://kenshin579.tistory.com/entry/Installing-Xen-on-Ubuntu-1004

Active Profiling

Dom0 controls the profiler. DomU green1 and green2 are active domains. 

># cp /usr/src/build/400/xen-4.0.1rc3/debian/build/result/boot/xen-syms-4.0.1-rc3  /boot/

># cp /usr/src/linux-source-2.6.31.12-xen/vmlinux /boot/vmlinux-2.6.31.12-xen

Copy two images to DomUs

># scp /boot/xen-syms-4.0.1-rc3 green1:/boot

># scp /boot/xen-syms-4.0.1-rc3 green2:/boot

># scp /boot/vmlinux-2.6.31.12-xen green1:/boot

># scp /boot/vmlinux-2.6.31.12-xen green2:/boot

># scp -rp /lib/modules/`uname -r` green1:/lib/modules

># scp -rp /lib/modules/`uname -r` green2:/lib/modules

Initialize

dom0># opcontrol --reset

dom2># opcontrol --reset

dom3># opcontrol --reset

Start OProfile in all domU

dom0># opcontrol --start-daemon --event=CPU_CLK_UNHALTED:6000 --xen=/boot/xen-syms-4.0.1-rc3 --vmlinux=/boot/vmlinux-2.6.31.12-xen  --active-domains=2,3

dom2># opcontrol --start --xen=/boot/xen-syms-4.0.1-rc3 --vmlinux=/boot/vmlinux-2.6.31.12-xen 

dom3># opcontrol --start --event=CPU_CLK_UNHALTED:6000 --xen=/boot/xen-syms-4.0.1-rc3 --vmlinux=/boot/vmlinux-2.6.31.12-xen 

Start profiling 

dom0># opcontrol --start

Run experiments to be profiled

dom2># ./loop

Stop profiling

dom0># opcontrol --stop

Shutdown OProfile daemon

dom0># opcontrol --shutdown

dom2># opcontrol --shutdown

dom3># opcontrol --shutdown

dom0># opreport -lt1

error: no sample files found: profile specification too strict 

I am not sure why this is the case. 

Passive Profiling

It seems like opcontrol doesn not have --passive-domains options. I have oprofile 0.9.6 version. 

dom0># opcontrol --reset

dom0># opcontrol --start-daemon --event=CPU_CLK_UNHALTED:6000 --xen=/boot/xen-syms-4.0.1-rc3 --vmlinux=/boot/vmlinux-2.6.31.12-xen  --passive-domains=2,3 --passive-images=/boot/vmlinux-2.6.31.12-xen,/boot/vmlinux-2.6.31.12-xen

It seems like the file, /dev/oprofile/passive_domains, is not there. What is the problem?

Solution is not known at the moment. Does anybody know?

dom0># opcontrol --start

Run experiments to be profiled

dom0># opcontrol --stop

dom0># opcontrol --shutdown

dom0># opreport -lt1

References

1. OProfile: Profiling in Linux for Fun and Profit, http://lbrandy.com/blog/2008/11/oprofile-profiling-in-linux-for-fun-and-profit/

2. OProfile on Xen, http://holypsycho.egloos.com/2952689

3. Controlling the profiler, http://oprofile.sourceforge.net/doc/controlling.html#eventspec

4. Xenoprof sourceforget.net, http://xenoprof.sourceforge.net

5. http://xenoprof.sourceforge.net/xenoprof_2.0.txt

6. Xen 4.0을 우분투 10.04서버 위에 설치하기, http://blog.relip.org/category/%EA%B0%80%EC%83%81%ED%99%94/Xen

7.