Video Tutorial How to Find TraceLine & Call Traceline with Inline ASM

Hexui Undetected CSGO Cheats Sinkicheat PUBG Cheat

Rake

Cesspool Admin
Administrator
Jan 21, 2014
12,118
78,998
2,392
How long you been coding/hacking?
5years
Coding Language
C++
Learn how to find and reverse engineer traceline in this IDA Pro Reverse Engineering tutorial. Then learn how to call the function with the code posted at the bottom using inline assembly.

How to Find Traceline / RayTrace - IDA Pro Reverse Engineering Tutorial


What is trace line?

Trace line is a function which takes two positions in 3d space and discovers if there is a collision in between them. It does this by drawing a virtual line from the source to the destination and on each specific increment it detects if a collision with another object has been detected

Video synopsis:
If we look at a assaultcube here I am looking at timber so we can see his name in the lower left hand corner when the walls in between us the name disappears so this is using a trace line function to detect that. There's many different ways to find this and there are many different types of these functions used in every 3d game because collision detection is a hugely important part of 3d game programming. So how do you find it? Anytime you're asking yourself that question you have to think logically like you were the game designer. How would I detect if my bullet was gonna hit someone when I pull the trigger?

Well inside my shoot function I would call trace line. How do we get that name in the lower left hand corner to pop up? We need to call trace line. So what you want to do is find things that you know and trace back to things you don't know so I'm gonna show you two ways to find it in assaultcube. If you haven't watched already I did a trigger bot tutorial what teaches you all about it and we do touch on trace line just a little bit and that video shows how you get the name in the lower left-hand corner.

So for this video we're gonna try the different way so we are going to start with our weapon ammo because we want to find our shoot function so to find the shoot function we're gonna start with our current weapon ammo and we're gonna do find out all rights to its address we'll shoot the gun we get the decrement ammo instruction so after we shoot our gun we decrease our ammo so that's probably in or after the gun shoot function so we are going to grab decrement ammo address and we're gonna go straight into Ida Pro.

Paste that in and f5 it I always f5 first get a top-down high-level view of what you're looking at so you can identify important things quickly and then later when you need to really dig down into it you can look at the assembly. So let's just say this is the gun shoot function which I'm pretty sure it is we see is a __this call function so that means it's a member function and that this is probably a vTable function. So let's just see if we could identify any of these things here. 0x50F4F4 we know this is the local player pointer let's see what else we can find.

We see our decrement ammo function doing the decrement instruction and it's v2 and it's indexing into the fifth element of the V 2 array. This is just how it's accessing that variable it's not technically an array what is v2 if we go up here v2 is right here and V 2 gets its value from the this pointer so v2 we're just gonna rename that to this to just a copy of this so when we're down here and we're looking at this to index 5 its indexing into the fifth for the fifth four byte variable in this object which we know is a weapon object...

Watch the video to learn more.

Here my aimbot which uses traceline

Here's my updated class method that checks if the player is visible to the local player and how I call it with inline ASM:

C++:
struct traceresult_s
{
    vec end;
    bool collided;
};

bool PlayerClass::IsVisible()
{
    DWORD traceLine = 0x048a310;
    traceresult_s traceresult;
    traceresult.collided = false;
    vec from = localPlayer->vHead;
    vec to = ent->vHead;

    __asm
    {
        push 0; bSkipTags
        push 0; bCheckPlayers
        push localPlayer
        push to.z
        push to.y
        push to.x
        push from.z
        push from.y
        push from.x
        lea eax, [traceresult]
        call traceLine;
        add esp, 36
    }
    return !traceresult.collided;
}
And then basically in my aimbot code I do this:

C++:
targets.clear();
        for each(PlayerClass p in playerVector)
        {
            if (p.ent == nullptr || p.ent->state != 0 || bTeamGame && p.ent->team == localPlayer->team)
            {
                continue;
            }

            if (!p.IsVisible())
            {
                continue;
            }
            targets.push_back(p);
        }
 
Last edited:

Liduen

Hacker
Dank Tier VIP
May 19, 2013
702
8,478
33
So this checks wether an entity is visible or not?

If so, I have a few questions :)
- How did you find this function?
- How did you figure out how traceresult_s has to look like?
- Will you do a tutorial on this?
 
Last edited:

Solaire

Respected Hacker
Dank Tier VIP
Dec 15, 2013
1,051
16,353
62
So this checks wether an entity is visible or not?

If so, I have a few questions :)
- How did you find this function?
- How did you figure out how traceresult_s has to look like?
- Will you do a tutorial on this?
Probably his answers:
1) It's in the source, and it's called in a lot of the gun functions, so it's not too hard to find a call to it.
2) Source code :p
3) (I'm not him, so this is his choice)
 

Rake

Cesspool Admin
Administrator
Jan 21, 2014
12,118
78,998
2,392
I've been trying to call this function without using inline ASM but I can't figure out the calling convention. It is not a member function so it's not thiscall. The assault cube source code has cdecl set as the default calling convention.

Here is the function declaration from the source code:
C++:
void TraceLine(vec from, vec to, dynent *pTracer, bool CheckPlayers, traceresult_s *tr, bool SkipTags)
When you look at the assembly *tr gets passed into EAX before calling the function. This doesn't follow any normal calling convention and that is why I used inline ASM. Was there a better alternative solution I could have used? Like I could write a wrapper for the function and in it I could push *tr into eax before calling a declspec naked function but wouldn't that be pointless if I already wrote the function inline assembly...

Is there any drawbacks to using inline ASM like this?
 

till0sch

Respected Hacker
Dank Tier VIP
Dank Tier Donator
Oct 14, 2012
1,104
12,593
51
Can only be __cdecl and nothing else because that's the only convention where the caller cleans the stack up which is the case if you look for calls at your trace line
 

till0sch

Respected Hacker
Dank Tier VIP
Dank Tier Donator
Oct 14, 2012
1,104
12,593
51
This guy has the same problem pretty much:
https://stackoverflow.com/questions/7692554/c-custom-calling-convention
and the answer by somebody was to create a wrapper what you already did.

This seems compiler related..


I'm not sure how they did their own calling convention but:
"The Watcom C/C++ compiler also uses the #pragma aux directive that allows the user to specify their own calling convention."


I always thought, though that AssaultCube was compiled in VS..
 

Rake

Cesspool Admin
Administrator
Jan 21, 2014
12,118
78,998
2,392
Ok I'm trying to make this video tutorial on how to find traceline, do you guys have any tips?

We all know it traces from vector A to vector B, checks if walls are hit or hits the boundary of the map, and usually stores the result in a struct

Here's my present tutorial in AC so far

Read breakpoint on your position
476 instructions access it
Look at each function and decide "Maybe traceline" or "definately not traceline"
Then look at your "maybe traceline" collection 1 by 1 and reverse it until you find traceline

Surely there must be a better way :p Anybody?
 

mambda

headass
Escobar Tier VIP
Trump Tier Donator
Jun 25, 2014
2,298
37,938
269
If there's anything that changes when you have a player directly in your sight, thats usually where i reverse from.

I.e. in AC, player names show up, right? Possibly check for a bool, or an index being changed from 0/false, to #/true
In CSGO, same thing basically, name and a red crosshair, doing that leads you to something that gets set as a direct result of a traceline like function, then you just work backwards
 

makane

Newbie
Full Member
May 23, 2016
5
353
1
First post! I've learned a lot coming here. The videos from yt channel are great (especially Rake's because he tries to explain everything.)

So far, I've made a GDI+ overlay esp and aimbot
GDI+ overlay = MSDN/stackoverflow
aimbot =
Rake's explanation for Calculating angles = https://guidedhacking.com/showthread.php?8165-The-Ultimate-CalcAngle-Thread
tut to understand the math! = https://www.unkn0wncheats.me/forum/counterstrike-global-offensive/137492-math-behind-hack-1-coding-better-aimbot-stop-using-calcangle.html

I could go on linking all the great info I've found by searching the internet (I've linked most of them in my source), but sometimes, I struggle to come up with creative ways to find stuff using cheat engine/ida on my own.

I've been trying to come up with a way to find traceline function (want to use it to check who is visible, so my aimbot doesn't just shoot at the closest enemy, which may be blocked by a wall)).

The unsuccessful method I've tried to use to find the traceline function was to see what changed up on the screen when I looked at an entity. (Thought it would lead to the traceline func)


1) Get the name of one of the entities and search for text in CE, in this case "Coward"


2) Move to another player and look at their name -> see change in CE, "Coward" changes to "New_B"


3) Get the address of this string (501c38). Find out what writes to this address. Grab the address and insert into IDA


4) Disassembled the function a bit. The variable "entity that player is looking at" is set by the function. We go into the function and ...


5) get to what seems like a function that checks the entity type and returns a player* if the type is valid. We go into another function that finds if the entity is on the screen(?)


6) from what I've decomplied, this function iterates through the entity list and finds if the entity is visible (returning result = 1 or 2) based on some weird matrix math in the function sub_4ff50.

I gave up at one point, and just got the traceline function address from rake's post
https://guidedhacking.com/showthread.php?6695-Calling-traceline-with-inline-ASM

but I didn't feel satisfied at all by doing this and want to go find this function myself. :(

Any tips/tricks? :/ I'll keep looking at the weird matrix math function, but I feel really stuck.
 
  • Like
Reactions: RenTec and bezier

Rake

Cesspool Admin
Administrator
Jan 21, 2014
12,118
78,998
2,392
superb reversing job so far!

First IDA pic is drawhud(), second is GetPlayerInCrosshair() and third is intersectclosest(). damn intersectclosest() looks exactly like traceline.



I don't think you ever would have traced backwards to find the TraceLine from this because TraceLine() and playerincrosshair() never get called from the same function and aren't member functions of the same class. But you coulda figured out the triggerbot from there :)
 

makane

Newbie
Full Member
May 23, 2016
5
353
1
Thanks a lot!
I realized a made a couple mistakes while reversing the parameters. (completely screwed up using a mattrix instead of a vector, lol).

Hmm...I still want to find traceline on my own (I don't want to give up yet!)

I was thinking that since traceline takes in the player's x position, I could find addresses that access the player's position, but the list is long in CE.

Would there be another (possibly easier) way to find trace line?
 
Last edited:

Rake

Cesspool Admin
Administrator
Jan 21, 2014
12,118
78,998
2,392
Would there be another (possibly easier) way to find trace line?
Find ammo decrement instruction, I believe the first function call above that instruction is a call to a function that then calls traceline, that's how I found it
 

makane

Newbie
Full Member
May 23, 2016
5
353
1
Rake;49997 said:
Find ammo decrement instruction, I believe the first function call above that instruction is a call to a function that then calls traceline, that's how I found it
Oh. Tracing from the decreasing ammo function seems a lot more logical than tracing through all the functions that access the player's position.


Found the decrease ammo function function and looked for the thiscall function. Should be in the weapon's vtable


And... woot, I have it. Took a while since my knowledge of reversing vtables is limited.

I'll continue reversing this function. Thanks a ton, Rake!
 
  • Like
Reactions: Rake

Rake

Cesspool Admin
Administrator
Jan 21, 2014
12,118
78,998
2,392
nice work!
Reversing vtable functions is easier than just some random function because you should have some basic knowledge of what the class variables might be and what kind of functionality the member functions should provide, alot of the member functions will touch the same member variable offsets so each piece you reverse will give you more information about the rest of the functions. fun stuff!
 

immortal.

Full Member
Mar 14, 2018
8
48
0
Hi yet again,

Well I'm working on my own AC hack atm and I essentially copy and pasted this code in this fashion (except I pass the pointer to the player to the function) and it doesn't seem to work (isPlayerVisible() always returns true). What I mean is that in my "aimbot loop" players that are not visible are added to my "good list" of players.

C++:
bool AimbotManager::isPlayerVisible(PlayerEnt * player)
{
    DWORD traceLine = 0x048a310;
    traceresult_s traceresult;
    traceresult.collided = false;
    Vec3 from = localPlayer->pos;
    Vec3 to = player->pos;
    from.z -= 0.2f;

    __asm
    {
        push 0; bSkipTags
        push 0; bCheckPlayers
        push localPlayer
        push to.z
        push to.y
        push to.x
        push from.z
        push from.y
        push from.x
        lea eax, [traceresult]
        call traceLine;
        add esp, 36
    }
    return !traceresult.collided;
}
This is my main aimbot loop where I check if the player is visible.

C++:
void AimbotManager::updateCurrentPlayers()
{
    currentPlayers.clear();
    playerList = *(EntList**)0x50F4F8;
    for (int i = 0; i < *numOfPlayers; i++)
    {
        if (playerList)
        {
            PlayerEnt * currentPlayer = playerList->ents[i];
            if (currentPlayer && IsValidEnt(currentPlayer))
            {
                if (localPlayer->team == currentPlayer->team || currentPlayer->health < 1 || !isPlayerVisible(currentPlayer))
                {
                    continue;
                }
                struct PlayerEntInfo playerEntInfo(calculateAngle(currentPlayer), getDistanceToPlayer(currentPlayer));
                currentPlayers.push_back(playerEntInfo);
            }
        }
    }
    std::sort(currentPlayers.begin(), currentPlayers.end(), ComparePlayerEntInfos());
}
Is it possible that the traceLine "0x048a310" static address is outdated?
 

Rake

Cesspool Admin
Administrator
Jan 21, 2014
12,118
78,998
2,392
No it's not outdated. Looking at your code everything looks perfect to me...not sure what the issue is. If you want you can zip up the entire project and send it to me and I can help you debug it...I won't be home the next couple day so I may not be able to look until later
 

immortal.

Full Member
Mar 14, 2018
8
48
0
No it's not outdated. Looking at your code everything looks perfect to me...not sure what the issue is. If you want you can zip up the entire project and send it to me and I can help you debug it...I won't be home the next couple day so I may not be able to look until later
Sorry for the late reply. Well I zipped up the VS project/solution if you want to take a look. To enable the aimbot hold right mouse, to toggle the shitty GDI ESP on or off use the F1 key. Build with VS2017 if you can, since the project demands that you have platform toolset v141 at least.

One day I'll learn function hooking so I can use OpenGL/DirectX instead of the flickering GDI.
 

Attachments

Not2EXceL

The rust BC is trash...wait no im a shit dev
Dank Tier Donator
Nobleman
Jan 20, 2013
130
1,743
3
Sorry for the late reply. Well I zipped up the VS project/solution if you want to take a look. To enable the aimbot hold right mouse, to toggle the shitty GDI ESP on or off use the F1 key. Build with VS2017 if you can, since the project demands that you have platform toolset v141 at least.

One day I'll learn function hooking so I can use OpenGL/DirectX instead of the flickering GDI.
double or even triple buffered gdi should not flicker if implemented properly. yeah its older and shit compared to ogl or directx but flickering is known with non buffered rendering and can be avoided
 
Attention! Before you post:

Read the How to Ask Questions Guide
99% of questions are answered in the Beginner's Guide, do it before asking a question.

No Hack Requests. Post in the correct section.  Search the forum first. Read the rules.

How to make a good post:

  • Fill out the form correctly
  • Tell us the game name & coding language
  • Post everything we need to know to help you
  • Ask specific questions, be descriptive
  • Post errors, line numbers & screenshots
  • Post code snippets using code tags
  • If it's a large project, zip it up and attach it

If you do not comply, your post may be deleted.  We want to help, please make a good post and we will do our best to help you.

Community Mods