第六章实验:虚拟化安全
实验一:使用LibVMI获取虚拟机相关信息
安装libvmi
背景知识
libvmi 库提供了对正在运行中的底层虚拟机的运行细节进行监视的功能。监视的功能是由观察内存细节,陷入硬件事件和读取 CPU 寄存器来完成的。这种方式被称作虚拟机自省 (virtual machine introspection)。
实验步骤
本实验使用 ubuntu20.04 进行安装。
仓库地址: libvmi/libvmi: The official home of the LibVMI project is at https://github.com/libvmi/libvmi.
安装文档: Setup — KVM-VMI 0.1 documentation
注意事项:
(1)根据 Option 2: Bare-metal setup 一节的内容安装。 (2)注意 KVM-VMI/kvm-vmi 中有较多的 submodule,若不使用安装文档中提供的带 –recursive 参数的下载命令,则需要手动补全这些 submodule,以免后续编译出错。
- 安装KVM-VMI
根据文档安装。
注意事项:
(1)KVM-VMI需要用kvmi-v7分支。
(2)在make之前需要额外执行以下指令(相关issue: Compile Failure: kvm-vmi/kvm/tools/objtool/.fixdep.o.cmd:1: *** Missing separator. Stop. · Issue #57 · KVM-VMI/kvm (github.com) ):
make distclean
make olddefconfig
./scripts/config --enable KVM
./scripts/config --enable KVM_INTEL
./scripts/config --enable KVM_AMD
./scripts/config --enable KVM_INTROSPECTION
./scripts/config --disable TRANSPARENT_HUGEPAGE
./scripts/config --enable REMOTE_MAPPING
./scripts/config --disable SYSTEM_TRUSTED_KEYS
./scripts/config --disable SYSTEM_REVOCATION_KEYS
- 安装QEMU
根据文档安装。
注意事项:
(1)需要关闭apparmor(或者修改其配置):
sudo service apparmor stop
- 在虚拟机中添加qemu指令
根据文档添加即可。
其中 http://libvirt.org/schemas/domain/qemu/1.0 这个网址为空,访问不到内容,具体说明在 libvirt: QEMU/KVM/HVF hypervisor driver 中,按照文档添加即可,无需担心。
- 安装libkvmi
根据文档安装即可。
- 安装LibVMI
根据文档安装即可。
应用 libvmi 的示例程序获取虚拟机信息
1) 打印 Windows 内核的 GUID 和 PE_HEADER;2)拦截并显示 CR3 事件;3)打印虚拟机中运行的进程列表。
实验步骤
先将 kvm-vmi/libvmi/etc/libvmi-example.conf 复制到 /etc/libvmi.conf,然后将虚拟机名字改为其中对应的名字。这里我们使用 Windows7 虚拟机(32位),所以就将虚拟机名字改为 win7,与配置文件对应(修改配置文件使其与虚拟机名字对应亦可):
win7 {
ostype = "Windows";
win_pdbase = 0x18;
win_pid = 0xb4;
win_tasks = 0xb8;
win_pname = 0x16c;
}
- 1)打印 Windows 内核的 GUID 和 PE_HEADER
使用 example 中的示例程序:
➜ build git:(5882bc2) ✗ examples/vmi-win-guid name win7 /tmp/introspector
Windows Kernel found @ 0x3c14000
Version: 32-bit Windows 7
PE GUID: 4ce78a09412000
PDB GUID: 684da42a30cc450f81c535b4d18944b12
Kernel filename: ntkrpamp.pdb
Multi-processor with PAE (version 5.0 and higher)
Signature: 17744.
Machine: 332.
# of sections: 22.
# of symbols: 0.
Timestamp: 1290242569.
Characteristics: 290.
Optional header size: 224.
Optional header type: 0x10b
Section 1: .text
Section 2: _PAGELK
Section 3: POOLMI
Section 4: POOLCODE
Section 5: .data
Section 6: ALMOSTRO
Section 7: SPINLOCK
Section 8: PAGE
Section 9: PAGELK
Section 10: PAGEKD
Section 11: PAGEVRFY
Section 12: PAGEHDLS
Section 13: PAGEBGFX
Section 14: PAGEVRFB
Section 15: .edata
Section 16: PAGEDATA
Section 17: PAGEKDD
Section 18: PAGEVRFC
Section 19: PAGEVRFD
Section 20: INIT
Section 21: .rsrc
Section 22: .reloc
- 2)拦截并显示 CR3 事件
使用 example 中的示例程序:
➜ build git:(5882bc2) ✗ examples/cr3-event-example win7 /tmp/introspector
Waiting for events...
CR3 write happened: Value=0x7ee0d480
CR3 write happened: Value=0x185000
CR3 write happened: Value=0x7ee0d480
CR3 write happened: Value=0x7ee0d220
CR3 write happened: Value=0x7ee0d480
CR3 write happened: Value=0x7ee0d220
CR3 write happened: Value=0x185000
CR3 write happened: Value=0x7ee0d480
CR3 write happened: Value=0x185000
CR3 write happened: Value=0x7ee0d480
CR3 write happened: Value=0x7ee0d220
CR3 write happened: Value=0x7ee0d480
CR3 write happened: Value=0x7ee0d220
CR3 write happened: Value=0x7ee0d480
......
- 3)打印虚拟机中运行的进程列表
使用 example 中的示例程序:
➜ build git:(5882bc2) ✗ examples/vmi-process-list -n win7 -s /tmp/introspector
LibVMI Suggestion: set win_ntoskrnl=0x3c14000 in libvmi.conf for faster startup.
LibVMI Suggestion: set win_kdbg=0x12ac28 in libvmi.conf for faster startup.
LibVMI Suggestion: set win_kdvb=0x83d3ec28 in libvmi.conf for faster startup.
Process listing for VM win7 (id=4)
[ 4] System (struct addr:85c3bc78)
[ 224] smss.exe (struct addr:975deae0)
[ 304] csrss.exe (struct addr:86d40768)
[ 352] wininit.exe (struct addr:92c8dd40)
[ 360] csrss.exe (struct addr:86d46030)
[ 400] winlogon.exe (struct addr:85cad420)
[ 444] services.exe (struct addr:86dca5e8)
[ 452] lsass.exe (struct addr:86e124e0)
[ 460] lsm.exe (struct addr:86e14828)
[ 564] svchost.exe (struct addr:86e47180)
[ 632] svchost.exe (struct addr:86e5c780)
[ 684] svchost.exe (struct addr:86e68568)
[ 808] svchost.exe (struct addr:86e35a58)
[ 848] svchost.exe (struct addr:86eb0030)
[ 912] audiodg.exe (struct addr:86ebf500)
[ 964] svchost.exe (struct addr:86ece030)
[ 1080] svchost.exe (struct addr:86ef2030)
[ 1200] dwm.exe (struct addr:86f26030)
[ 1212] explorer.exe (struct addr:86f289f0)
[ 1256] spoolsv.exe (struct addr:86f34030)
[ 1296] svchost.exe (struct addr:86f51d40)
[ 1308] taskhost.exe (struct addr:86f59868)
[ 1428] svchost.exe (struct addr:86fa8030)
[ 1824] rundll32.exe (struct addr:8703cb18)
[ 1844] rundll32.exe (struct addr:8703ed40)
[ 1864] rundll32.exe (struct addr:870412e8)
[ 660] SearchIndexer. (struct addr:870c8958)
[ 1232] SearchProtocol (struct addr:870efb88)
[ 1412] SearchFilterHo (struct addr:87106c70)
[ 1288] wmpnetwk.exe (struct addr:87116cb0)
调用LibVMI的接口实现进程信息获取
**1)修改示例程序中的 cr3-event-example.c;2)修改 示例程序中的 process-list.c,根据需要获取进程、线程信息。 ** libvmi 的 api 文档:https://libvmi.com/api/
实验步骤
我们来看看有哪些事件支持被监控:
// libvmi/events.h
#define VMI_EVENT_INVALID 0
#define VMI_EVENT_MEMORY 1 /**< Read/write/execute on a region of memory */
#define VMI_EVENT_REGISTER 2 /**< Read/write of a specific register */
#define VMI_EVENT_SINGLESTEP 3 /**< Instructions being executed on a set of VCPUs */
#define VMI_EVENT_INTERRUPT 4 /**< Interrupts being delivered */
#define VMI_EVENT_GUEST_REQUEST 5 /**< Guest-requested event */
#define VMI_EVENT_CPUID 6 /**< CPUID event */
#define VMI_EVENT_DEBUG_EXCEPTION 7 /**< Debug exception event */
#define VMI_EVENT_PRIVILEGED_CALL 8 /**< Privileged call (ie. SMC on ARM) */
#define VMI_EVENT_DESCRIPTOR_ACCESS 9 /**< A descriptor table register was accessed */
#define VMI_EVENT_FAILED_EMULATION 10 /**< Emulation failed when requested by VMI_EVENT_RESPONSE_EMULATE */
#define VMI_EVENT_DOMAIN_WATCH 11 /**< Watch create/destroy events */
- 1)修改示例程序中的 cr3-event-example.c
来看看 vmi_event 中包含了什么,我们可以从中选择想要监控的信息打印出来:
// libvmi/events.h
struct vmi_event {
/* CONST IN */
uint32_t version; /**< User should set it to VMI_EVENTS_VERSION */
/* CONST IN */
vmi_event_type_t type; /**< The specific type of event */
/**
* IN/OUT/RESPONSE
*
* The VMM maintained SLAT ID. Can be specified when registering mem_event (IN).
* On an event report (OUT) specifies the active SLAT ID on the vCPU.
* Iff VMI_EVENT_RESPONSE_SLAT_ID is set (RESPONSE), switch the vCPU to this VMM pagetable ID.
*
* Note: on Xen this corresponds to the altp2m_idx.
*/
uint16_t slat_id;
/**
* RESPONSE
*
* The VMM should switch to this SLAT ID on the occurance of the next event.
* Iff VMI_EVENT_RESPONSE_NEXT_SLAT_ID is set.
*
* Note: on Xen this corresponds to the altp2m_idx and it also enables MTF singlestepping.
* The altp2m switch automatically happens in the singlestep handler in Xen after a single
* instruction is executed.
*/
uint16_t next_slat_id;
/**
* CONST IN
*
* An open-ended mechanism allowing a library user to
* associate external data to the event.
* Metadata assigned to this pointer at any time (prior to
* or following registration) is delivered to the callback,
* for each matching event. The callback is also free to
* modify in any way. The library user assumes all memory
* management for this referenced data.
*/
void *data;
/**
* CONST IN
*
* The callback function that is invoked when the relevant is observed.
*/
event_callback_t callback;
/* OUT */
uint32_t vcpu_id; /**< The VCPU relative to which the event occurred. */
/**
* Reserved for future use
*/
uint32_t _reserved[7];
union {
reg_event_t reg_event;
mem_access_event_t mem_event;
single_step_event_t ss_event;
interrupt_event_t interrupt_event;
privcall_event_t privcall_event;
cpuid_event_t cpuid_event;
debug_event_t debug_event;
descriptor_event_t descriptor_event;
watch_domain_event_t watch_event;
};
/*
* Note that the following pointers assume compiler compatibility
* ie. if you compiled a 32-bit version of LibVMI it will be
* incompatable with 64-bit tools and vice verse.
*/
union {
/**
* OUT
*
* Snapshot of some VCPU registers when the event occurred
*/
union {
x86_registers_t *x86_regs;
arm_registers_t *arm_regs;
};
/**
* RESPONSE
*
* Read data to be sent back with VMI_EVENT_RESPONSE_SET_EMUL_READ_DATA
*/
emul_read_t *emul_read;
/**
* RESPONSE
*
* Instruction buffer to be sent back with VMI_EVENT_RESPONSE_SET_EMUL_INSN
*/
emul_insn_t *emul_insn;
};
};
因为我们这里选择了监控寄存器,所以再来看看 reg_event_t 中包含什么:
// libvmi/events.h
typedef struct {
/**
* CONST IN
*
* Register for which write event is configured.
* Hypervisors offering register events tend to
* have a limited number available for monitoring.
* These registers tend to be those defined as
* 'sensitive register instructions' by Popek and
* Goldberg, meaning that the registers trigger
* a VMEXIT, trap, or equivalent.
*
* Note for MSR events on Xen: up to Xen 4.7 only MSR_ALL is supported.
* Starting with Xen 4.8 the user has the option to subscribe to specific
* MSR events, or to continue using MSR_ALL. However, in this case MSR_ALL
* only corresponds to common MSRs that are defined by LibVMI in libvmi.h.
* To subscribe to MSR events that are NOT defined by LibVMI, the user can specify
* MSR_UNDEFINED here and then set the specific MSR index in the 'msr' field
* below.
*/
reg_t reg;
/**
* CONST IN
*
* Event filter: callback triggers IFF register==<equal>
*/
reg_t equal;
/**
* CONST IN
*
* IFF set to 1, events are delivered asynchronously and
* without pausing the originating VCPU
* Default : 0. (i.e., VCPU is paused at time of event delivery).
*/
uint8_t async;
/**
* CONST IN
*
* IFF set to 1, events are only delivered if the written
* value differs from the previously held value.
* Default : 0. (i.e., All write events are delivered).
*/
uint8_t onchange;
/**
* CONST IN
*
* Type of register event being monitored.
* Hypervisors offering register events do so only for those that trigger a
* VMEXIT or similar trap. This predominantly means that only write events
* are supported by the corresponding LibVMI driver
*/
vmi_reg_access_t in_access;
/**
* OUT
*
* Type of register access that triggered the event
*/
vmi_reg_access_t out_access;
uint32_t _pad;
/**
* OUT
*
* Register value read or written
*/
reg_t value;
/**
* OUT
*
* Previous value of register (only for CR0/CR3/CR4/MSR)
*/
reg_t previous;
/**
* CONST IN/OUT
*
* MSR register operations only
*
* CONST IN: Starting from Xen 4.8 the user can use this field to specify an
* MSR index to subscribe to when the MSR is not formally defined by LibVMI.
*
* OUT: holds the specific MSR for which the event occurred
* when the user registered with MSR_ALL.
* Unused for other register event types.
*/
uint32_t msr;
} reg_event_t;
选择其中一些感兴趣的数据,并修改 call back 函数:
// cr3-event-example.c
...
event_response_t my_callback(vmi_instance_t vmi, vmi_event_t *event)
{
(void)vmi;
printf("CR3 write happened: Value=0x%"PRIx64"\n", event->reg_event.value);
printf("\tvcpu_id=0x%"PRIx64"\n", event->vcpu_id);
printf("\trbx=0x%"PRIx64"\n", event->x86_regs->rbx);
printf("\tcr0=0x%"PRIx64"\n", event->x86_regs->cr0);
printf("\tprevious CR3 value=0x%"PRIx64"\n", event->reg_event.previous);
return VMI_EVENT_RESPONSE_NONE;
}
...
int main (int argc, char **argv)
{
...
vmi_event_t my_event = {0};
my_event.version = VMI_EVENTS_VERSION;
my_event.type = VMI_EVENT_REGISTER;
my_event.callback = my_callback;
my_event.reg_event.reg = CR3;
my_event.reg_event.in_access = VMI_REGACCESS_W;
...
}
重新 make,然后运行:
➜ build git:(5882bc2) ✗ examples/cr3-event-example win7 /tmp/introspector
Waiting for events...
CR3 write happened: Value=0x7f9740a0
vcpu_id=0x0
rbx=0x83d77c00
cr0=0x8001003b
previous CR3 value=0x185000
CR3 write happened: Value=0x185000
vcpu_id=0x0
rbx=0x83d77c00
cr0=0x8001003b
previous CR3 value=0x7f9740a0
CR3 write happened: Value=0x7f9740a0
vcpu_id=0x0
rbx=0x83d77c00
cr0=0x8001003b
previous CR3 value=0x185000
CR3 write happened: Value=0x185000
vcpu_id=0x0
rbx=0x83d77c00
cr0=0x8001003b
previous CR3 value=0x7f9740a0
CR3 write happened: Value=0x7f9740a0
vcpu_id=0x0
rbx=0x83d77c00
cr0=0x8001003b
previous CR3 value=0x185000
......
- 2)修改 process-list.c
在windows中,EPROCESS 结构属于内核的执行体层,包含了进程的资源相关信息诸如句柄表、虚拟内存、安全、调试、异常、创建信息、I/O转移统计以及进程计时等。32位 win7 系统 EPROCESS 结构如下:
kd> dt _EPROCESS
ntdll!_EPROCESS
+0x000 Pcb : _KPROCESS
+0x098 ProcessLock : _EX_PUSH_LOCK
+0x0a0 CreateTime : _LARGE_INTEGER
+0x0a8 ExitTime : _LARGE_INTEGER
+0x0b0 RundownProtect : _EX_RUNDOWN_REF
+0x0b4 UniqueProcessId : Ptr32 Void
+0x0b8 ActiveProcessLinks : _LIST_ENTRY
+0x0c0 ProcessQuotaUsage : [2] Uint4B
+0x0c8 ProcessQuotaPeak : [2] Uint4B
+0x0d0 CommitCharge : Uint4B
+0x0d4 QuotaBlock : Ptr32 _EPROCESS_QUOTA_BLOCK
+0x0d8 CpuQuotaBlock : Ptr32 _PS_CPU_QUOTA_BLOCK
+0x0dc PeakVirtualSize : Uint4B
+0x0e0 VirtualSize : Uint4B
+0x0e4 SessionProcessLinks : _LIST_ENTRY
+0x0ec DebugPort : Ptr32 Void
+0x0f0 ExceptionPortData : Ptr32 Void
+0x0f0 ExceptionPortValue : Uint4B
+0x0f0 ExceptionPortState : Pos 0, 3 Bits
+0x0f4 ObjectTable : Ptr32 _HANDLE_TABLE
+0x0f8 Token : _EX_FAST_REF
+0x0fc WorkingSetPage : Uint4B
+0x100 AddressCreationLock : _EX_PUSH_LOCK
+0x104 RotateInProgress : Ptr32 _ETHREAD
+0x108 ForkInProgress : Ptr32 _ETHREAD
+0x10c HardwareTrigger : Uint4B
+0x110 PhysicalVadRoot : Ptr32 _MM_AVL_TABLE
+0x114 CloneRoot : Ptr32 Void
+0x118 NumberOfPrivatePages : Uint4B
+0x11c NumberOfLockedPages : Uint4B
+0x120 Win32Process : Ptr32 Void
+0x124 Job : Ptr32 _EJOB
+0x128 SectionObject : Ptr32 Void
+0x12c SectionBaseAddress : Ptr32 Void
+0x130 Cookie : Uint4B
+0x134 Spare8 : Uint4B
+0x138 WorkingSetWatch : Ptr32 _PAGEFAULT_HISTORY
+0x13c Win32WindowStation : Ptr32 Void
+0x140 InheritedFromUniqueProcessId : Ptr32 Void
+0x144 LdtInformation : Ptr32 Void
+0x148 VdmObjects : Ptr32 Void
+0x14c ConsoleHostProcess : Uint4B
+0x150 DeviceMap : Ptr32 Void
+0x154 EtwDataSource : Ptr32 Void
+0x158 FreeTebHint : Ptr32 Void
+0x160 PageDirectoryPte : _HARDWARE_PTE_X86
+0x160 Filler : Uint8B
+0x168 Session : Ptr32 Void
+0x16c ImageFileName : [15] UChar
+0x17b PriorityClass : UChar
+0x17c JobLinks : _LIST_ENTRY
+0x184 LockedPagesList : Ptr32 Void
+0x188 ThreadListHead : _LIST_ENTRY
......
与 EPROCESS 对应,ETHREAD 结构记录了线程中相关的信息。32位 win7 系统 ETHREAD 结构如下:
kd> dt _ETHREAD
ntdll!_ETHREAD
+0x000 Tcb : _KTHREAD
+0x200 CreateTime : _LARGE_INTEGER
+0x208 ExitTime : _LARGE_INTEGER
+0x208 KeyedWaitChain : _LIST_ENTRY
+0x210 ExitStatus : Int4B
+0x214 PostBlockList : _LIST_ENTRY
+0x214 ForwardLinkShadow : Ptr32 Void
+0x218 StartAddress : Ptr32 Void
+0x21c TerminationPort : Ptr32 _TERMINATION_PORT
+0x21c ReaperLink : Ptr32 _ETHREAD
+0x21c KeyedWaitValue : Ptr32 Void
+0x220 ActiveTimerListLock : Uint4B
+0x224 ActiveTimerListHead : _LIST_ENTRY
+0x22c Cid : _CLIENT_ID
......
我们根据这些信息,修改 process-list.c:
int main (int argc, char **argv)
{
...
// 添加变量
vmi_pid_t pid = 0, ppid = 0;
unsigned long tasks_offset = 0, pid_offset = 0, name_offset = 0, pdbase_offset = 0;
...
/* init the offset values */
if (VMI_OS_LINUX == vmi_get_ostype(vmi)) {
if ( VMI_FAILURE == vmi_get_offset(vmi, "linux_tasks", &tasks_offset) )
goto error_exit;
if ( VMI_FAILURE == vmi_get_offset(vmi, "linux_name", &name_offset) )
goto error_exit;
if ( VMI_FAILURE == vmi_get_offset(vmi, "linux_pid", &pid_offset) )
goto error_exit;
} else if (VMI_OS_WINDOWS == vmi_get_ostype(vmi)) {
if ( VMI_FAILURE == vmi_get_offset(vmi, "win_tasks", &tasks_offset) )
goto error_exit;
if ( VMI_FAILURE == vmi_get_offset(vmi, "win_pname", &name_offset) )
goto error_exit;
if ( VMI_FAILURE == vmi_get_offset(vmi, "win_pid", &pid_offset) )
goto error_exit;
// 添加获取win_pdbase的步骤
if ( VMI_FAILURE == vmi_get_offset(vmi, "win_pdbase", &pdbase_offset) )
goto error_exit;
} else if (VMI_OS_FREEBSD == vmi_get_ostype(vmi)) {
tasks_offset = 0;
if ( VMI_FAILURE == vmi_get_offset(vmi, "freebsd_name", &name_offset) )
goto error_exit;
if ( VMI_FAILURE == vmi_get_offset(vmi, "freebsd_pid", &pid_offset) )
goto error_exit;
}
...
/* walk the task list */
while (1) {
current_process = cur_list_entry - tasks_offset;
/* Note: the task_struct that we are looking at has a lot of
* information. However, the process name and id are burried
* nice and deep. Instead of doing something sane like mapping
* this data to a task_struct, I'm just jumping to the location
* with the info that I want. This helps to make the example
* code cleaner, if not more fragile. In a real app, you'd
* want to do this a little more robust :-) See
* include/linux/sched.h for mode details */
/* NOTE: _EPROCESS.UniqueProcessId is a really VOID*, but is never > 32 bits,
* so this is safe enough for x64 Windows for example purposes */
vmi_read_32_va(vmi, current_process + pid_offset, 0, (uint32_t*)&pid);
procname = vmi_read_str_va(vmi, current_process + name_offset, 0);
// 根据偏移获取各种进程、线程信息
uint32_t pdbase = 0;
vmi_read_32_va(vmi, current_process + pdbase_offset, 0, (uint32_t*)&pdbase);
vmi_read_32_va(vmi, current_process + 0x140, 0, (uint32_t*)&ppid);
vmi_read_addr_va(vmi, current_process + 0x188, 0, (addr_t*)&ThreadListHead);
ThreadListHead = ThreadListHead - 0x268;
int64_t CreateTime = 0;
vmi_read_64_va(vmi, ThreadListHead + 0x200, 0, &CreateTime);
uint32_t Cid = 0;
vmi_read_64_va(vmi, ThreadListHead + 0x22c, 0, (uint32_t*)&Cid);
if (!procname) {
printf("Failed to find procname\n");
goto error_exit;
}
/* print out the process name */
if((pid > 5) && (strcmp(procname, "svchost.exe") != 0)) {
printf("[%5d] %s (struct addr:%"PRIx64"), pdbase:%"PRIx64", ppid:%5d\n", pid, procname, current_process, pdbase, ppid);
// 添加我们需要的输出语句
printf("ThreadListHead:%"PRIx64"\n", ThreadListHead);
printf("thread createtime:%ld\n", CreateTime);
printf("Cid:%5d\n", Cid);
}
...
}
输出结果:
➜ build git:(5882bc2) ✗ examples/vmi-process-list -n win7 -s /tmp/introspector
LibVMI Suggestion: set win_ntoskrnl=0x3c14000 in libvmi.conf for faster startup.
LibVMI Suggestion: set win_kdbg=0x12ac28 in libvmi.conf for faster startup.
LibVMI Suggestion: set win_kdvb=0x83d3ec28 in libvmi.conf for faster startup.
Process listing for VM win7 (id=4)
[ 224] smss.exe (struct addr:975deae0), pdbase:7ee0d020, ppid: 4
ThreadListHead:975de808
thread createtime:133255839485781250
Cid: 224
[ 304] csrss.exe (struct addr:86d40768), pdbase:7ee0d060, ppid: 296
ThreadListHead:85cc87b0
thread createtime:133255839501718750
Cid: 304
[ 352] wininit.exe (struct addr:92c8dd40), pdbase:7ee0d0a0, ppid: 296
ThreadListHead:85cb3770
thread createtime:133255839501875000
Cid: 352
[ 360] csrss.exe (struct addr:86d46030), pdbase:7ee0d040, ppid: 344
ThreadListHead:85cad828
thread createtime:133255839502968750
Cid: 360
[ 400] winlogon.exe (struct addr:85cad420), pdbase:7ee0d0c0, ppid: 344
ThreadListHead:86c07d48
thread createtime:133255839503125000
Cid: 400
[ 444] services.exe (struct addr:86dca5e8), pdbase:7ee0d080, ppid: 352
ThreadListHead:86e0f030
thread createtime:133255839506718750
Cid: 444
[ 452] lsass.exe (struct addr:86e124e0), pdbase:7ee0d0e0, ppid: 352
ThreadListHead:86e20030
thread createtime:133255839507656250
Cid: 452
[ 460] lsm.exe (struct addr:86e14828), pdbase:7ee0d100, ppid: 352
ThreadListHead:86e144a0
thread createtime:133255839507187500
Cid: 460
[ 1200] dwm.exe (struct addr:86f26030), pdbase:7ee0d280, ppid: 808
ThreadListHead:86f26658
thread createtime:133255839526718750
Cid: 1200
......
读者可以根据示例自行添加需要获取的信息。上述两个结构的具体信息读者可以自行查阅或自行调试获取。
参考6.3小节,获取并输出Linux内核系统调用表sys_call_table的内容
背景知识
符号(Symbols),就是kernel中的变量(Variable Name)或函数名称(Function Name)。在Linux中,System.map文件是内核符号名称及其对应内存地址的映射表。它在内核构建过程中创建,并位于内核源代码树的根目录中(通常是/linux/System.map)。System.map文件由内核的崩溃转储分析工具和一些性能分析工具使用,以将内核地址映射到它们对应的符号名称。
实验步骤
这一个实验我们以 ubuntu14.04 为例。
安装 ubuntu14.04 系统以及修改虚拟机 XML 文件的过程不在此赘述。
虚拟机安装完成后,我们将它的 System.map 文件复制到宿主机 boot 目录下(其他路径也可以),检查一下 sys_call_table 符号是否存在:
➜ libvmi git:(5882bc2) ✗ cat /boot/System.map-3.13.0-24-generic | grep sys_call_table
ffffffff81801400 R sys_call_table
ffffffff81809cc0 R ia32_sys_call_table
然后修改 /etc/libvmi.conf 文件:
ubuntu14.04 {
sysmap = "/boot/System.map-3.13.0-24-generic";
ostype = "Linux";
}
接着用 map-symbol 程序查看这一块内存的内容:
➜ libvmi git:(5882bc2) ✗ build/examples/map-symbol ubuntu14.04 sys_call_table /tmp/introspector
00000000| 80 9e 1b 81 ff ff ff ff 20 9f 1b 81 ff ff ff ff ........ .......
00000010| b0 89 1b 81 ff ff ff ff 20 6b 1b 81 ff ff ff ff ........ k......
00000020| 90 e9 1b 81 ff ff ff ff c0 e9 1b 81 ff ff ff ff ................
00000030| a0 e9 1b 81 ff ff ff ff 10 e9 1c 81 ff ff ff ff ................
00000040| 00 90 1b 81 ff ff ff ff 30 86 01 81 ff ff ff ff ........0.......
00000050| d0 10 18 81 ff ff ff ff 20 fe 17 81 ff ff ff ff ........ .......
00000060| e0 f2 17 81 ff ff ff ff a0 bb 07 81 ff ff ff ff ................
00000070| 30 a9 07 81 ff ff ff ff 10 6a 72 81 ff ff ff ff 0........jr.....
00000080| c0 c8 1c 81 ff ff ff ff c0 9f 1b 81 ff ff ff ff ................
00000090| 70 a0 1b 81 ff ff ff ff 10 a5 1b 81 ff ff ff ff p...............
000000a0| d0 a5 1b 81 ff ff ff ff 60 7b 1b 81 ff ff ff ff ........`{......
000000b0| f0 2d 1c 81 ff ff ff ff 30 e0 1c 81 ff ff ff ff .-......0.......
000000c0| 80 9a 09 81 ff ff ff ff e0 1d 18 81 ff ff ff ff ................
000000d0| f0 22 18 81 ff ff ff ff 60 ab 17 81 ff ff ff ff ."......`.......
000000e0| a0 35 17 81 ff ff ff ff 80 fa 2b 81 ff ff ff ff .5........+.....
000000f0| 60 02 2c 81 ff ff ff ff e0 fa 2b 81 ff ff ff ff `.,.......+.....
00000100| c0 76 1d 81 ff ff ff ff 80 75 1d 81 ff ff ff ff .v.......u......
00000110| 40 be 07 81 ff ff ff ff f0 f4 08 81 ff ff ff ff @...............
00000120| 40 ad 06 81 ff ff ff ff 60 62 07 81 ff ff ff ff @.......`b......
00000130| 30 b1 06 81 ff ff ff ff b0 e3 07 81 ff ff ff ff 0...............
00000140| d0 ac 1b 81 ff ff ff ff a0 2d 60 81 ff ff ff ff .........-`.....
00000150| 60 31 60 81 ff ff ff ff 40 31 60 81 ff ff ff ff `1`.....@1`.....
......
实验二:嵌套虚拟化环境实验
配置并重新编译KVM虚拟化环境,开启嵌套虚拟化模式,运行启动嵌套虚拟机
实验步骤
检查L0 KVM的嵌套虚拟化支持是否开启:
cat /sys/module/kvm_intel/parameters/nested
如果显示Y
说明已经开启。
创建、启动L1、L2虚拟机的步骤详见:第一章实验
注意:使用qemu-system-x86_64
命令启动虚拟机时,应加入-cpu host
参数,以便上层vCPU继承下层CPU(或vCPU)的VMX特性。
在L0层KVM中捕获L1层KVM对嵌套虚拟机VMCS的写操作
实验思路
在KVM嵌套虚拟化实现中,L1、L2虚拟机实际都由L0物理机启动。
但在逻辑上(即从L1的视角),L2虚拟机由L1虚拟机中的KVM维护、创建、启动。
因此,L0、L1分别维护了如下VMCS数据结构:
VMCS名称 | 实际维护者 | 逻辑上的host | 逻辑上的guest |
---|---|---|---|
vmcs01 | L0 | L0 | L1 |
vmcs12 | L1 | L1 | L2 |
vmcs02 | L0 | L0 | L2 |
shadow_vmcs12 | L0 | L1 | L2 |
当L2发生VM exit,需要进入L1 KVM处理,并返回L2,该过程步骤如下:
- L2 VM exit,控制流进入L0 KVM
- L0 KVM读取vmcs02数据,将guest部分内容更新至shadow_vmcs12
- L0 KVM将shadow_vmcs12推送至L1虚拟机中维护的vmcs12
- L0 KVM切换VMCS到vmcs01,控制流进入L1 KVM
- L1 KVM识别到L2虚拟机的陷入并处理,继续执行L2虚拟机,该指令被L0捕获后,控制流回到L0KVM
- L0 KVM从L1虚拟机中读取到已经被L1 KVM维护过的vmcs12,存入shadow_vmcs12
- L0 KVM将shadow_vmcs12中guest相关数据更新至vmcs02
- L0 KVM切换VMCS到vmcs02,控制流回到L2虚拟机
因此,只需在L1处理L2 VM exit的前(步骤3、步骤4之间)后(步骤6、步骤7之间),对vmcs12分别进行读取、对比,即可捕获L1层KVM对嵌套虚拟机VMCS的写操作。
其中,在L1处理L2 VM exit前(步骤3、步骤4之间)加入代码如下:
record_vmcs12_data = vmcs12->guest_cr0;
在L1处理L2 VM exit后(步骤6、步骤7之间)加入代码如下:
if (record_vmcs12_data != vmcs12->guest_cr0)
{
printk("[cpt6-exp2] before L1 run, guest_cr0=%llx\n",record_vmcs12_data);
printk("[cpt6-exp2] after L1 run, guest_cr0=%llx\n",vmcs12->guest_cr0);
}
其中record_vmcs12_data
为全局变量。
实验步骤
使用补丁文件exp2/cpt6-exp2-kernel.patch修改Linux内核源码,编译、安装:
make -j
sudo rmmod kvm_intel
sudo insmod kernel-source/arch/x86/kvm/kvm-intel.ko
依次启动L1、L2虚拟机
打开dmesg可以看到L1 KVM对L2虚拟机CR0寄存器的修改,如图:
