Tutorial My perspective on converting world coordinates to screen coordinates

Hexui Undetected CSGO Cheats Sinkicheat PUBG Cheat

Nomade

Newbie
Meme Tier VIP
Aug 23, 2017
154
5,528
15
Game Name
Any
Anticheat
N/A
Tutorial Link
N/A
How long you been coding/hacking?
~1.8yrs
Coding Language
C/C++

Your game might use a different matrix orientation. It will be either column major or row major.
Knowledge of matrix multiplication is required.


Hello, recently I've made my first ESP and I went from knowing nothing about the view matrix to knowing enough for an ESP. I'm gonna describe the process.

I learned a lot from these two tutorials:
Rake's tutorial on how to find the view matrix
Mambda's tutorial on the view matrix and ws2()

If you want a geometry perspective on matrix multiplication I highly recommend this video:


Goal: to make a function that you input a world coordinate(3D) and it outputs the corresponding screen coordinate(2D). Let's call such function w2s().

I'll not tell you what the view matrix is or describes because it's not required for an ESP. Also, I don't really know :FeelsBadMan:.

w2s() has 2 steps:
  1. Multiply the world coordinate vector with the view matrix.
  2. Do some scaling to the result of step 1 so it fits nicely on the display.
Step 1
Consider the 3D world coordinate a 4D vector. How? Like this: (x, y, z) -> (x, y, z, 1).
Multiply the 1x4 world position matrix with the 4x4 view matrix. Note: A 3D vector can be viewed like a 1x3 matrix, just like a 4D vector can be viewed like a 1x4 matrix.
The output of this step is a 1x4 matrix(4D vector). Let's call it the screen vector.

Step 2(The scaling step might be different between games, so if this doesn't work print the values to the screen and determine the right way to scale)
screen[0] will eventually be the x screen position(after scaling).
screen[1] will eventually be the y screen position(after scaling).
screen[2] is ignored because a monitor doesn't have depth.
screen[3] is an important value. To be honest I don't know what it is exactly but here's some intuition: screen[0] and screen[1] will vary quite a bit if you change your position/angles. When you divide screen[0] / screen[3] the result should be a value between [-1, 1](the same applies for screen[1]). So think of it as a scaling factor that constrains screen[0] and screen[1] to the range [-1, 1].

Ok, now screen[0] and screen[1] are values in the range [-1, 1](after dividing both by screen[3]). We should convert the range to [0, 1] because it'll be useful later on. You don't need any fancy math for that. Think of an equation that converts a value between these ranges, otherwise look at the spoiler tag :fleep: .
C++:
// Pick any
x/2 + 0.5
0.5*x + 0.5
0.5*(x + 1)

The final step is to multiply screen[0] with the width of the game's window and screen[1] with the height of the game's window. That's it, now you have a screen coordinate.

Potential problem
You may ask yourself: What if I apply w2s() to a world coordinate behind me(not in FOV)? The answer is: You'll get garbage values.
The solution is simple: After doing the matrix multiplication check if screen[3] is a value less than 0, if it is the world position is not in your FOV.

Finding the view matrix
As you just saw the view matrix is a matrix of order 4(4x4 matrix). In memory it's probably an array of 16 floats so it's easily recognizable.
Where do you start from? Well, I use Rake's method(at least that's where I saw it first) of searching for +1.0 when looking up and -1.0 when looking down(some games may not allow you to fully look up/down so the value might be 0.98 or something). This will get you down to a few values. Then look at a different angle and delete values which are either +1 or -1 because it should have been somewhere in between. If there're more than a few values try respawning or changing the level with the hope that some values will point to invalid memory. Now you must check value by value to determine which ones are potential good matrices.


Eliminating bad matrices:
Is this a good matrix?
1601691980544.png

I can guarantee you this matrix will not work, not because it has big values(this might actually be an issue if the values are really big, not in this case though) but because when you do the matrix multiplication 'w' will always be 1. Why? Because we're doing the following to calculate 'w':
C++:
float w = world_pos[0] * view_matrix[0 * 4 + 3] + world_pos[1] * view_matrix[1 * 4 + 3] + world_pos[2] * view_matrix[2 * 4 + 3] + view_matrix[3 * 4 + 3];
which will result in:
C++:
float w = world_pos[0] * 0 + world_pos[1] * 0 + world_pos[2] * 0 + view_matrix[3 * 4 + 3];
You see... 'w' will always be 1. This cannot happen because we divide x and y by w to scale them to the range [-1, +1] and if 'w' is always 1 it'll not scale.

So, eliminate all values which look like this, then individually test all remaining matrices(there should only be a few)

Possible w2s() function implementation(You just learned how to build your own so this is just to give you an idea):
C++:
bool w2s(float* world_pos, float* screen_pos)
{
    float* view_matrix = (float*)(base + view_matrix_offset);

    float x = world_pos[0] * view_matrix[0 * 4 + 0] + world_pos[1] * view_matrix[1 * 4 + 0] + world_pos[2] * view_matrix[2 * 4 + 0] + view_matrix[3 * 4 + 0];
    float y = world_pos[0] * view_matrix[0 * 4 + 1] + world_pos[1] * view_matrix[1 * 4 + 1] + world_pos[2] * view_matrix[2 * 4 + 1] + view_matrix[3 * 4 + 1];
    float w = world_pos[0] * view_matrix[0 * 4 + 3] + world_pos[1] * view_matrix[1 * 4 + 3] + world_pos[2] * view_matrix[2 * 4 + 3] + view_matrix[3 * 4 + 3];

    if (w < 0.0f)
        return false;

    x /= w;
    y /= w;

    screen_pos[0] = (x + 1) * 0.5f * viewport.Width;
    screen_pos[1] = (y + 1) * 0.5f * viewport.Height;

    return true;
}

 

Attachments

Last edited:

XdarionX

Dying Light Hacker
Dank Tier VIP
Dank Tier Donator
Mar 30, 2018
865
24,608
114
Great job! at least game guru has some purpose ;)
 
  • Like
Reactions: Nomade
Attention! Before you post:

Read the How to Ask Questions Guide
99% of questions are answered in the Beginner's Guide, do it before asking a question.

No Hack Requests. Post in the correct section.  Search the forum first. Read the rules.

How to make a good post:

  • Fill out the form correctly
  • Tell us the game name & coding language
  • Post everything we need to know to help you
  • Ask specific questions, be descriptive
  • Post errors, line numbers & screenshots
  • Post code snippets using code tags
  • If it's a large project, zip it up and attach it

If you do not comply, your post may be deleted.  We want to help, please make a good post and we will do our best to help you.

Community Mods