Detours not cleaning the stack correctly

Question

I'm trying to hook a function using Detours for the first time. I'm new to reversing software and to hooking, so I may have missed something big here.

I'm trying to use this function as a hook:

typedef  int (__thiscall* func_type)(LPVOID*, LPVOID*, DWORD, BOOL);
LPVOID hookaddr = 0;
 int __fastcall testhook(LPVOID* pThis, void* _EDX, LPVOID* object, DWORD hp, BOOL self)
{
    std::cout << "Hooked"  << std::endl;

func_type originalFunc = (func_type)hookaddr;

return originalFunc(pThis, object, hp, self);
}

The problem lies in the fact that the hooked function seems to follow __thiscall convention and Detours doesn't seem to be allowing this? I tried mixing __thiscall and __stdcall/__fastcall conventions in my injected dll, but couldn't get anything working. I either end up with a wrong ecx value or a crash due to an invalid esp.

Any idea what I could try here?

Stud · Answer

I have found the solution to this problem, which is quite simple. I got confused with the hooked function signature and the __fastcall trick to get a non member function to work with the __thiscall convention. In my initial post, the two first arguments in the function signatures are the one passed using ecx and edx, but I forgot about one of the pointer passed using the stack. The correct function definition is the following:

int __fastcall testhook(LPVOID* pThis, void* _EDX, LPVOID* object, LPVOID* object2, DWORD hp, BOOL self)
{
    std::cout << "Hooked"  << std::endl;

func_type originalFunc = (func_type)hookaddr;

return originalFunc(pThis, object, object2, hp, self);
}

As the callee is responsible for cleaning the stack, providing an incorrect number of parameters lead to a stack corruption (in my case, one of the parameter was still on the stack after my function returns).

Detours not cleaning the stack correctly

One Answer

Add your own answers!

Ask a Question