//display Hello World
&dwUnused, //doesn't matter what buffer we use
0, //because we aren't writing anything to it
It uses project properties from this tutorial to make it smaller. It's alot easier to convert when you don't have excess code.
We can have the compiler do most of the work for us, by going to the project Property Pages -> Configuration Properties -> C/C++ -> Output Files -> Assembler Output. I generally choose the "Assembly, Machine Code and Source (/FAcs)" option. After rebuilding the project once you selected this option, the compiler will generate .cod files in the solution\project\x64\release folder. These can be opened in Visual Studio.
The compiler will generate something like this
; ; Listing generated by Microsoft (R) Optimizing Compiler Version 17.00.61030.0
0005f 48 83 c4 38 add rsp, 56 ; 00000038H
00063 c3 ret 0
Since there is no native Masm project type in Visual Studio, you have to use Visual C++. Create a new project and use Visual C++ -> Win32 Console Application (since we're converting a console app). Click Next, check the "Empty project" checkbox, and click finish. The project should be completely empty. Select the project name in the solution explorer and then, from the main menu, go to Project -> Build Customizations... (it's Custom Build Rules... in 2008). Check the masm checkbox and click OK.
Set it to 64-bit from the Configuration Manager, click the dropdown box under Platform. Set New platform to x64, and click OK, and click Close.
In case that wasn't clear, here is good tutorial on setting up masm in 64-bit.
From What's a Creel?
Make sure to copy the project linker properties from the C++ project to this one.
Linker -> Input -> Addtional Dependencies kernel32.lib
Linker -> Input -> Ignore All Default Libraries Yes (/NODEFAULTLIB)
Linker -> Advanced -> Entry Point main
Linker -> Manifest File -> Generate Manifest No (/MANIFEST:NO)
Linker -> Advanced -> Randomized Base Address No (/DYNAMICBASE:NO)
Linker -> Advanced -> Fixed Base Address Yes (/FIXED)
Linker -> Advanced -> Merge Sections .rdata=.text
To add assembly files to the project, add a new item. Click the C++ file type, and give it a .asm extension instead of a .cpp extension. In this case, I named it main.asm.
If you didn't add masm before you added a .asm file. Right-click the .asm file go to Properties -> Configuration Properties -> General -> Item Type. Set it to Microsoft Macro Assembler and click OK.
Take a look at the main.cod again, we'll be using it as a template for most of the conversion. You'll notice it does not have
#define STD_INPUT_HANDLE ((DWORD)-10)
#define STD_OUTPUT_HANDLE ((DWORD)-11)
Can be translated in masm as:
STD_INPUT_HANDLE equ -10
STD_OUTPUT_HANDLE equ -11
Imports are easy, you use EXTRN keyword, then the symbol name from the .lib, followed by :PROC. Imports from the window's 64-bit libraries often begin with __imp_.
I prefer adding the align 16 in at the begining of each segment. This will align the data/instruction, under it, on a 16 byte boundary.
The .cod is a bit confusing when it comes to read only data, it has a CONST ENDS but no CONST SEGMENT to begin the segment. The g_szHelloWorld is the name. DB (Define Byte) is the data type. 'Hello Masm64 World', 00H is the data, a ascii string with a null terminator.
The pdata, and xdata comdats are used with exception handling. Since we aren't using any, we can ignore them.
Next up is the code section which is also known as the .text section in the PE. Following the example from the .cod, each procedure (function) gets its own [URL="stackoverflow.com/questions/1834597/what-is-the-comdat-section-used-for"]comdat[\URL].
dwUnused$ = 64 Here dwUnused$ is basically set as a alias for 64. So when we come across QWORD PTR dwUnused$[rsp], it really means, QWORD PTR [rsp+64]. This is where dwUnused is stored on the stack.
In order to determine how much stack space we need, we start with 20h (32) for shadow space. Then we find the function that takes the most parameters (excluding floating point parameters). If we have any functions with five or more parameters, we subtract four from it, because those are passed via registers. Multiply it by eight and add the total bytes of local variables. Finally, add 8 to keep the stack 16-byte aligned.
We start with 20h (32). Both WriteFile, and ReadFile take five parameters, so we're going to need to add 8 bytes. Add another 4 bytes for dwUnused. Finally add 0Ch (12) to get 38h (56).
Every .asm file needs to end with END. This can cause funky errors if you put it anywhere else.