Using Custom Function Calling Convention in IDA Pro

It's possible to "define" custom calling convention in IDA Pro disassembly database (at least in version 6.1). For example, the following function uses ax register and the stack to pass parameters.

assignI16toI64a proc near

 pDstI64= word ptr  4

 push    bp
 mov     bp, sp
 mov     bx, [bp+pDstI64]
 mov     [bx+I64.mWords.mWord0], ax ; <-- this is one of the parameter
 mov     [bx+I64.mWords.mWord1], 0
 mov     [bx+I64.mDwords.mDword1], 0
 mov     ax, bx
 leave
 retn    2
assignI16toI64a endp

How do we "inform" IDA Pro about the calling convention? Look at this hint from IDA Pro help.

IDA supports the user-defined calling convention. In this calling convention, the user can explicitly specify the locations of arguments and the return value. For example:

        int __usercall func<ebx>(int x, int y<esi>);

denotes a function with 2 arguments: the first argument is passed on the stack and the second argument is passed in the ESI register and the return value is stored in the EBX register.

Let's put this knowledge to the function above. Go to the "Set Function Type"  command (the default is "y" keyboard button). Set the function type as follows:

I64* __usercall assignI16toI64a<ax>(short Src<ax>, I64 *pDstI64)

Now, we have the custom function declaration. Let's see how the "auto commenting" works in the call to this function:

     push    ax                              ; pDstI64
     xor     ax, ax                          ; Src
     call    assignI16toI64a

As you can see, the function parameter "auto commenting" works as expected, marking the ax register as one of the parameter (as intended).