'iret not returning to protected mode unexpectedly
I am writing an interrupt handler in x86. A process that traps, first builds up a trap frame that contains its register context. I added a image where you can check the contents of the stack containing the trap frame.
After the interrupt handling code finishes, the kernel returns these following lines of code, restoring the register context that was saved in the trap frame from above. The registers cs, esp, eip, and ss are restored by the iret instruction.
.globl trapret
trapret:
popal
popl %gs
popl %fs
popl %es
popl %ds
addl $0x8, %esp # trapno and errcode
iret
What I expect to happen after trapret, is to start executing at the entry point at eip. However, after calling iret, for some reason I enter virtual 8086 mode. Since my original eip value is added with the cs register in v8086 mode, I cant enter the line of code that I want to.
For example, if I set the eip value in the trap frame to 0x1d0 (entry to the routine I want to run), the register context looks like this after calling iret.
The eip value is set to what I wanted (0x1d0) but since I entered v8086 mode, the actual instruction address that is being read is (0x1b0(from cs) + 0x1d0 = 0x380).
I read about the operation of iret, and it seems like if the VM bit of eflags are set to 0 and PE bit in CR0 is 1, iret will return to protected mode. The eflags value I stored to the trap frame is 0x2 and the PE bit is set to 1 in CR0 when I execute iret. Still, I get this strange behavior when calling iret. What can I do to make iret return to protected mode and run the line of code I intend to run?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|