'call GetConsoleScreenBufferInfoEx trashes my stack return address [duplicate]
I am using latest Visual Studio 2022 Preview to learn X64 assembly language. My assembler is the builtin MASM64. The application type is a simple Windows 11 X64 console. the following code causes an exception:
; WinAPI libraries.
INCLUDELIB kernel32.lib ; Console.
; WinAPI type definitions.
HANDLE TYPEDEF QWORD
ULONG TYPEDEF DWORD
.CONST
MyInvalidValue EQU -1 ; Error detection.
.DATA
ALIGN 4
CONSOLE_SCREEN_BUFFER_INFOEX STRUCT
ALIGN 4
cbSize ULONG SIZEOF CONSOLE_SCREEN_BUFFER_INFOEX
ALIGN 2
dwSizeX WORD MyInvalidValue
dwSizeY WORD MyInvalidValue
dwCursorPositionX WORD MyInvalidValue
dwCursorPositionY WORD MyInvalidValue
wAttributes WORD MyInvalidValue
srWindowL WORD MyInvalidValue
srWindowT WORD MyInvalidValue
srWindowR WORD MyInvalidValue
srWindowB WORD MyInvalidValue
dwMaximumWindowSizeX WORD MyInvalidValue
dwMaximumWindowSizeY WORD MyInvalidValue
wPopupAttributes WORD MyInvalidValue
ALIGN 4
bFullscreenSupported DWORD MyInvalidValue
COLORREF DWORD 16 DUP (MyInvalidValue)
CONSOLE_SCREEN_BUFFER_INFOEX ENDS
MyBufferS CONSOLE_SCREEN_BUFFER_INFOEX {}
ALIGN 8
MyStdOutH HANDLE MyInvalidValue
.CODE
; External WinAPI procedures.
EXTERNDEF ExitProcess :PROC
EXTERNDEF GetStdHandle :PROC
EXTERNDEF GetConsoleScreenBufferInfoEx:PROC
main PROC
sub rsp, 4 * 8 ; Shadow stack for 4 parameters.
call MyGetConsoleHandlesProcedure
call MySetConsoleSizeProcedure
add rsp, 5 * 8 ; Restores stack and alignment.
xor ecx, ecx ; Exits with no error code.
call ExitProcess
main ENDP
MyGetConsoleHandlesProcedure PROC ; Gets console handle.
mov ecx, 0FFFFFFF5h ; STD_OUTPUT_HANDLE nStdHandle.
call GetStdHandle
mov MyStdOutH, rax ; Saves handle.
ret
MyGetConsoleHandlesProcedure ENDP
MySetConsoleSizeProcedure PROC ; Sets size.
mov rcx, MyStdOutH ; hConsoleOutput.
lea rdx, MyBufferS ; lpConsoleScreenBufferInfoEx.
call GetConsoleScreenBufferInfoEx
ret
MySetConsoleSizeProcedure ENDP
END
The specific line that causes the Access violation exception is
call GetConsoleScreenBufferInfoEx
After carefull debugging I found out the reason. This call overwrites the stack return address so the ret instructions pops a wrong return address from the stack. This is a minimum viable program to show the bug. Obviously the full program has full error detection after every call and there are no errors... Here is a screenshot of the stack before this call: stack before the call As can be seen in the registers view, the stack is properly aligned before the call. And here is a screenshot of the stack after this call: stack after the call As can be seen, the return address was overwritten with all zeros! What am I doing wrong?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
