some code wrong with cr3 filter or i have not understand cr3 filter mechanism

[crackzhazha]

Accroding to the Intel® 64 and IA-32 Architectures Software Developer’s Manual.pdf section 35.2.7.6 IA32_RTIT_CR3_MATCH MSR Intel says Bits 63:5 hold the CR3 address value to match, bits 4:0 are reserved to 0.

But in your code at simple-pt.c line 774 function set_cr3_filter

static void set_cr3_filter(void *arg)
{
	u64 val;

	if (pt_rdmsrl_safe(MSR_IA32_RTIT_CTL, &val) < 0)
		return;
	if ((val & TRACE_EN) && pt_wrmsrl_safe(MSR_IA32_RTIT_CTL, val & ~TRACE_EN) < 0)
		return;
	if (pt_wrmsrl_safe(MSR_IA32_CR3_MATCH, *(u64 *)arg) < 0)
		pr_err("cpu %d, cannot set cr3 filter\n", smp_processor_id());
	if ((val & TRACE_EN) && pt_wrmsrl_safe(MSR_IA32_RTIT_CTL, val) < 0)
		return;
}

You haven't set the low 5bit of arg(the value of cr3) to 0, this may cause general-protection fault (#GP)
We know when low 5 bit is 0 ,the mask is 0xffffffffffffffe0
hex(0b1111111111111111111111111111111111111111111111111111111111100000)=0xffffffffffffffe0L
so this is my code for set_cr3_filter,you can ignore the code for logging.

static void set_cr3_filter_fix(void *arg)
{
        u64 val;

        if (pt_rdmsrl_safe(MSR_IA32_RTIT_CTL, &val) < 0)
                return;
        if ((val & TRACE_EN) && pt_wrmsrl_safe(MSR_IA32_RTIT_CTL, val & ~TRACE_EN) < 0)
                return;
        pr_err("now arg: %p,before set_cr3_filter",*(u64 *)arg);
        pr_err("simple-pt:Cpu %d Ready to set_cr3_filter: cr3:%p",smp_processor_id(),(*(u64 *)arg )&0xffffffffffffffe0 ) ;
        if (pt_wrmsrl_safe(MSR_IA32_CR3_MATCH, (*(u64 *)arg )&0xffffffffffffffe0 ) < 0)
                pr_err("cpu %d, cannot set cr3 filter\n", smp_processor_id());
        if ((val & TRACE_EN) && pt_wrmsrl_safe(MSR_IA32_RTIT_CTL, val) < 0)
                return;
}

And i have another question, after I perform this patch. I can only get trace log for my specific process in ring0 code,but can not get any log for my specific process in ring3 code. I wonder know it's my code error or i have not understand the mechanism of cr3 filter? Could you help figure out this question?
I want to use cr3 filter to trace an specific process both ring0 and cr3 code. thanks all.

[andikleen]

bit 5 should never be set because the page table is 4K page aligned.

Likely you need to boot with "nopti" on newer kernels, CR3 filter is incompatible with page table separation

[crackzhazha]

Sorry to bother you again,I roll back to your code. But when I don't set the low 5bit to 0, i can not set cr3_match MSR , this is my code for log the process of set cr3 filter ```c static void set_cr3_filter_log(void *arg) { u64 val; u64 cr3_match; if (pt_rdmsrl_safe(MSR_IA32_RTIT_CTL, &val) < 0) return; if ((val & TRACE_EN) && pt_wrmsrl_safe(MSR_IA32_RTIT_CTL, val & ~TRACE_EN) < 0) return; pr_err("now arg: %p,before set_cr3_filter",*(u64 *)arg); pr_err("simple-pt:Cpu %d Ready to set_cr3_filter: cr3:%p",smp_processor_id(),(*(u64 *)arg ) ) ; if (pt_wrmsrl_safe(MSR_IA32_CR3_MATCH, (*(u64 *)arg ) ) < 0) pr_err("cpu %d, cannot set cr3 filter\n", smp_processor_id()); pt_rdmsrl_safe(MSR_IA32_CR3_MATCH,&cr3_match); pr_err("now cr3_match: %p,after set_cr3_filter",cr3_match); if ((val & TRACE_EN) && pt_wrmsrl_safe(MSR_IA32_RTIT_CTL, val) < 0) return; } ``` after apply this code, You can see cr3_match MSR can not be set. It will be 0. My test environment is ` ➜ simple-pt git:(master) ✗ uname -a Linux pt-linux 4.13.0-45-generic #50~16.04.1-Ubuntu SMP Wed May 30 11:18:27 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux ➜ simple-pt git:(master) ✗ cat /etc/issue Ubuntu 16.04.4 LTS \n \l ➜ simple-pt git:(master) ✗ dmesg | grep 'Kernel/User page tables isolation' ➜ simple-pt git:(master) ✗ ` it seems the kernel have not apply pti (KPTI)? I wonder know what's wrong with my environment? Can you help me? Thanks a lot! 2018-06-26 5:35 GMT+08:00 Andi Kleen <[email protected]>:

…

bit 5 should never be set because the page table is 4K page aligned. Likely you need to boot with "nopti" on newer kernels, CR3 filter is incompatible with page table separation — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALEgf_MUSwd31z1VceDdU--NE2nfkhlSks5uAVecgaJpZM4U2ITx> .

[crackzhazha]

After i close pti ， i found that my fix code run correct in my environment. ```c static void set_cr3_filter_log(void *arg) { u64 val; u64 cr3_match; if (pt_rdmsrl_safe(MSR_IA32_RTIT_CTL, &val) < 0) return; if ((val & TRACE_EN) && pt_wrmsrl_safe(MSR_IA32_RTIT_CTL, val & ~TRACE_EN) < 0) return; pr_err("now arg: %p,before set_cr3_filter",*(u64 *)arg); pr_err("simple-pt:Cpu %d Ready to set_cr3_filter: cr3:%p",smp_processor_id(),(*(u64 *)arg )&0xffffffffffffffe0 ) ; if (pt_wrmsrl_safe(MSR_IA32_CR3_MATCH, (*(u64 *)arg ) &0xffffffffffffffe0 ) < 0) pr_err("cpu %d, cannot set cr3 filter\n", smp_processor_id()); pt_rdmsrl_safe(MSR_IA32_CR3_MATCH,&cr3_match); pr_err("now cr3_match: %p,after set_cr3_filter",cr3_match); if ((val & TRACE_EN) && pt_wrmsrl_safe(MSR_IA32_RTIT_CTL, val) < 0) return; } ``` why my cr3 is not 4k page align? [image: image.png] but hopefully,it can run correctly right now! But i have to change my code, Can you check your code in different environment again zhazha crack <[email protected]> 于2018年6月26日周二 上午8:39写道：

…

Sorry to bother you again,I roll back to your code. But when I don't set the low 5bit to 0, i can not set cr3_match MSR , this is my code for log the process of set cr3 filter ```c static void set_cr3_filter_log(void *arg) { u64 val; u64 cr3_match; if (pt_rdmsrl_safe(MSR_IA32_RTIT_CTL, &val) < 0) return; if ((val & TRACE_EN) && pt_wrmsrl_safe(MSR_IA32_RTIT_CTL, val & ~TRACE_EN) < 0) return; pr_err("now arg: %p,before set_cr3_filter",*(u64 *)arg); pr_err("simple-pt:Cpu %d Ready to set_cr3_filter: cr3:%p",smp_processor_id(),(*(u64 *)arg ) ) ; if (pt_wrmsrl_safe(MSR_IA32_CR3_MATCH, (*(u64 *)arg ) ) < 0) pr_err("cpu %d, cannot set cr3 filter\n", smp_processor_id()); pt_rdmsrl_safe(MSR_IA32_CR3_MATCH,&cr3_match); pr_err("now cr3_match: %p,after set_cr3_filter",cr3_match); if ((val & TRACE_EN) && pt_wrmsrl_safe(MSR_IA32_RTIT_CTL, val) < 0) return; } ``` after apply this code, You can see cr3_match MSR can not be set. It will be 0. My test environment is ` ➜ simple-pt git:(master) ✗ uname -a Linux pt-linux 4.13.0-45-generic #50~16.04.1-Ubuntu SMP Wed May 30 11:18:27 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux ➜ simple-pt git:(master) ✗ cat /etc/issue Ubuntu 16.04.4 LTS \n \l ➜ simple-pt git:(master) ✗ dmesg | grep 'Kernel/User page tables isolation' ➜ simple-pt git:(master) ✗ ` it seems the kernel have not apply pti (KPTI)? I wonder know what's wrong with my environment? Can you help me? Thanks a lot! 2018-06-26 5:35 GMT+08:00 Andi Kleen ***@***.***>: > bit 5 should never be set because the page table is 4K page aligned. > > Likely you need to boot with "nopti" on newer kernels, CR3 filter is > incompatible with page table separation > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#20 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/ALEgf_MUSwd31z1VceDdU--NE2nfkhlSks5uAVecgaJpZM4U2ITx> > . >

[andikleen]

The lower 5 bits are the PCID, which is used in PTI kernels.

simple-pt should probably check and warn about it, although right now it will just be checked by the WRMSR. Could also be masked out, but usually the filtering is not very useful with PTI anyways because you will only see kernel code.

[andikleen]

Should mask out the PCID in CR3