Visual Studio Code Question posted in
View the official documentation.

Visual Studio Code – Why MSVC compiler ignore my code and call stdclib function

I use Visual Studio 2022 with v143 platform toolset

Have that code (dll):

#include <Windows.h>

unsigned char* buff;

BOOL APIENTRY DllMain( HMODULE hModule, DWORD  ul_reason_for_call, LPVOID lpReserved )
{
    int len = 1000;
    while (len--)
        *buff++ = 0;
}

MSVC project settings:
Compiler:
Disable Security Check (/GS-)

Linker:
/NODEFAULTLIB
/ENTRY:"DllMain"

But compiler replace my code to do call memset() in stdclib
Listing dllmain.asm

_DllMain@12 PROC                    ; COMDAT

; 7    : {

    push    esi

; 8    :     int len = 1000;

    mov esi, DWORD PTR ?buff@@3PAEA     ; buff
    push    1000                    ; 000003e8H
    push    0
    push    esi
    call    _memset
    add esp, 12                 ; 0000000cH
    add esi, 1000               ; 000003e8H
    mov DWORD PTR ?buff@@3PAEA, esi     ; buff

; 9    :     while (len--)
; 10   :         *buff++ = 0;
; 11   : }

    xor eax, eax
    pop esi
    ret 12                  ; 0000000cH
_DllMain@12 ENDP

When I try compile exe file is fine, but can’t compile dll because linker ends with error "error LNK2001: unresolved external symbol _memset".

Why compiler do it? How to disable it?

Update:
I found solution
by adding volatile local var
volatile unsigned char* p = (unsigned char*)buff;

#include <Windows.h>

unsigned char* buff;

BOOL APIENTRY DllMain( HMODULE hModule, DWORD  ul_reason_for_call, LPVOID lpReserved )
{
    int len = 1000;
    volatile unsigned char* p = (unsigned char*)buff;
    while (len--)
        *p++ = 0;
}

_DllMain@12 PROC

; 19   :     int len = 1000;
; 20   :     volatile unsigned char* p = (unsigned char*)buff;

    mov ecx, DWORD PTR ?buff@@3PAEA     ; buff
    mov eax, 1000               ; 000003e8H
$LL2@DllMain:

; 21   :     while (len--)
; 22   :         *p++ = 0;

    mov BYTE PTR [ecx], 0
    inc ecx
    sub eax, 1
    jne SHORT $LL2@DllMain

; 23   : }

    ret 12                  ; 0000000cH
_DllMain@12 ENDP
Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I found solution
    by adding volatile local var volatile unsigned char* p = (unsigned char*)buff;

    #include <Windows.h>

unsigned char* buff;

BOOL APIENTRY DllMain( HMODULE hModule, DWORD  ul_reason_for_call, LPVOID lpReserved )
{
    int len = 1000;
    volatile unsigned char* p = (unsigned char*)buff;
    while (len--)
        *p++ = 0;
}

    _DllMain@12 PROC

; 19   :     int len = 1000;
; 20   :     volatile unsigned char* p = (unsigned char*)buff;

    mov ecx, DWORD PTR ?buff@@3PAEA     ; buff
    mov eax, 1000               ; 000003e8H
$LL2@DllMain:

; 21   :     while (len--)
; 22   :         *p++ = 0;

    mov BYTE PTR [ecx], 0
    inc ecx
    sub eax, 1
    jne SHORT $LL2@DllMain

; 23   : }

    ret 12                  ; 0000000cH
_DllMain@12 ENDP

  2. 0

    Optimizing C++ compilers will often recognize loops that do the equivalent of memset and memcpy and replace the code with an appropriate call. In some cases, they might even inline the code. This is a good thing for performance, as the implementations that come with the compiler (or the standard library it uses) are highly optimized.

    It’s not clear whether you actually want to disable the optimization or whether you’re assuming that’s what you have to do to resolve the linker error.

    In C++, you commonly compile for a "hosted" environment, which means that standard library is available. Some compilers also support a "freestanding" mode, which means the program cannot use the standard library (and there may be other limitations as well). Freestanding is commonly used for embedded systems.

    Even in freestanding mode, C++ compilers may generate calls to memcpy and memset. Even with optimizations disabled. So finding a way to disable the optimization in this example might not solve the entire problem.

    The reason the call to memset is not resolved in this example is that the /NODEFAULTLIB option told the linker not to consider functions from libraries it normally would have checked.

    Option 1: Remove /NODEFAULTLIB. The linker will resolve the call against the standard library. On Windows, there exist both static and DLL versions of the standard library. You can select which you want, but, if you’re making a DLL, you usually want the DLL version of the standard library.

    Option 2: If /NODEFAULTLIB is required to prevent libraries other than the standard library from being considered, then you can still tell the linker to consider the standard library.

    Option 3: If your DLL must not use any version of the standard library, provide your own definition of memset (and memcpy if necessary). You can write the obvious implementations, or you make wrappers around WinAPI calls like ZeroMemory and CopyMemory.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search